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Abstract  of  Dissertation  Presented  to  the  Graduate  School 
of  the  University  of  Florida  in  Partial  Fulfillment  of  the 
Requirements  for  the  Degree  of  Doctor  of  Philosophy 

VARIABLES  INFLUENCING  RISK  SENSITIVITY  IN  ADULT  HUMANS 

By 

Cynthia  J.  Pietras 
December,  2000 

Chair:  Dr.  Timothy  D.  Hackenberg,  Ph.D. 

Major  Department:  Psychology 

Three  experiments  examined  risky  choice  in  adult  humans  with  procedures  similar  to  those 
used  to  study  risky  choice  in  nonhumans.  These  experiments  were  designed  both  to  investigate  the 
generality  of  risky  choice  shown  in  previous  research  with  humans,  and  to  determine  whether  some 
of  the  discrepancies  that  exist  between  risky  choice  in  humans  and  nonhumans  are  due  to  species 
differences  or  to  methodological  differences  in  how  risky  choice  is  typically  assessed.  In  all 
experiments,  subjects  were  given  extensive  exposure  to  contingencies  and  outcomes  were  real 
money.  In  Experiments  1 and  2,  the  choice  outcomes  were  fixed  and  variable  reinforcer  amounts, 
and  fixed  and  variable  reinforcer  delays,  respectively.  The  form  of  the  variable  distribution  was 
manipulated  across  conditions  to  determine  how  distributional  characteristics  influenced  choice.  In 
both  experiments,  choice  was  unaffected  by  the  form  of  the  variable  distribution.  Consistent  with 
previous  research  with  humans,  choice  was  predominantly  risk  averse  in  Experiment  1 and  risk 
averse  or  risk  neutral  in  Experiment  2.  In  Experiment  3,  risky  choice  was  examined  under 
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conditions  designed  to  model  energy-budget  manipulations  conducted  by  behavioral  ecologists  with 
nonhumans.  Subjects  were  presented  with  repeated  choices  between  fixed  and  variable  point 
amounts.  Positive  and  negative  energy-budgets  were  simulated  by  manipulating  the  number  of 
points  that  had  to  be  accumulated  within  a block  of  trials  for  points  to  be  exchanged  for  money. 

The  results  were  generally  consistent  with  previous  energy-budget  research  and  with  the 
predictions  of  an  optimal  foraging  model  known  as  the  energy-budget  rule:  Choice  was  risk  averse 
when  energy  budgets  were  positive  and  usually  risk  prone  when  energy  budgets  were  negative. 
Local  choice  patterns  were  also  generally  consistent  with  the  predictions  of  a dynamic  optimization 
model.  The  results  of  Experiment  3 thus  demonstrated  the  generality  of  energy-budget  effects,  and 
suggested  that  risky  choice  in  humans  may  be  similar  to  that  shown  in  nonhumans  when  choice  is 
assessed  under  comparable  experimental  conditions.  The  contributions  of  procedural  differences  to 
discrepancies  in  risky  choice  shown  across  species,  and  the  benefits  of  developing  laboratory 
procedures  that  can  be  used  by  researchers  from  a variety  of  scientific  disciplines  are  discussed. 
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INTRODUCTION 


Humans  and  other  animals  are  often  confronted  with  choices  that  have  uncertain  or 
variable  outcomes.  For  example,  an  animal  might  be  faced  with  a choice  between  foraging  in  a 
patch  that  consistently  yields  a moderate  quantity  of  food  or  foraging  in  a patch  that  yields  either  a 
high  or  low  quantity  of  food  probabilistically.  A gambler  might  be  faced  with  a choice  between  a 
bet  with  high  odds  and  a small  payoff,  and  a bet  with  low  odds  and  a large  payoff. 

The  term  risk  has  been  used  to  refer  to  environmental  variability.  Preference  for  fixed 
(i.e.,  constant)  and  variable  alternatives  has  been  described  as  risk  sensitivity  (Kacelnik  & Bateson, 
1 996).  (This  definition  of  risk  is  to  be  distinguished  from  definitions  that  refer  to  choice  between 
options  with  potentially  harmful  consequences).  Risk  sensitivity  may  take  one  of  three  forms.  If  a 
fixed  option  is  preferred  to  a variable  option,  behavior  is  said  to  be  risk  averse.  If  a variable  option 
is  preferred  to  a fixed  option,  behavior  is  said  to  be  risk  prone  (Stephens  & Krebs,  1986).  Finally, 
if  no  strong  preference  for  either  option  occurs,  behavior  is  said  to  be  risk  neutral  or  indifferent. 

Risky  choice  in  humans  and  nonhumans  has  been  investigated  by  researchers  from  a 
variety  of  scientific  disciplines  including  economics,  psychology,  anthropology,  and  behavioral 
ecology.  This  research  has  generally  shown  that  behavior  is  sensitive  to  environmental  variability. 
An  overview  of  this  research  reveals,  however,  an  apparent  discrepancy  between  human  and 
nonhuman  risk  sensitivity.  Specifically,  humans  tend  to  show  greater  risk  aversion  than  do 
nonhumans. 

It  is  difficult  to  relate  patterns  of  risk  sensitivity  shown  in  humans  and  nonhumans, 
however,  because  the  procedures  used  to  assess  choice  vary  substantially  from  one  another.  Some 
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of  these  procedural  differences  (discussed  in  more  detail  below)  include  the  method  of  presenting 
choice  options  (i.e.,  presented  verbally  or  actually  experienced),  the  number  of  trials  over  which 
choice  is  assessed  (repeated  choices  or  one-shot  choice),  the  nature  of  the  choice  outcome  (real  or 
hypothetical),  the  form  of  the  variable  distribution  (e.g.,  exponential  or  bivalued),  and  the 
dimension  that  is  variable  (amount  or  delay).  The  extent  to  which  these  procedural  variables 
influence  risky  choice  or  contribute  to  human  and  nonhuman  differences  in  risk  sensitivity  is 
unclear.  Therefore,  one  aim  of  the  present  research  was  to  investigate  choice  in  humans  under 
conditions  more  comparable  to  those  used  with  nonhumans. 

Although  the  factors  responsible  for  the  different  patterns  of  risk  sensitivity  shown  in 
research  with  nonhumans  have  not  been  precisely  identified,  behavioral  ecologists  have  shown  that 
risk  sensitivity  in  many  species  varies  reliably  with  changes  in  the  organism’s  energy  budget. 
Energy  budget  refers  to  the  energy  status  of  an  organism  in  relation  to  its  energy  requirements 
(Bateson  & Kacelnik,  1998).  In  certain  species,  individuals  must  accumulate  sufficient  energy 
reserves  while  foraging  to  survive  a period  in  which  foraging  is  impossible  (e.g.,  overnight).  If 
reserves  are  high  or  the  mean  rate  of  food  intake  is  sufficient  to  exceed  the  energy  requirement,  the 
energy  budget  is  positive.  If  reserves  are  low  or  the  mean  rate  of  food  intake  is  insufficient  to  meet 
the  energy  requirement,  the  energy  budget  is  negative.  A number  of  studies  conducted  by 
behavioral  ecologists  (reviewed  below)  have  shown  that  risk  sensitivity  varies  as  a function  of 
energy  budget  (e.g.,  Caraco,  1983;  Caraco,  Martindale  & Whittam,  1980).  These  studies  have 
shown  that  choice  is  risk  averse  when  the  energy  budget  is  positive  and  risk  prone  when  the  energy 
budget  is  negative. 

Although  the  investigation  of  energy-budget  effects  has  become  one  of  the  most  promising 
approaches  to  the  analysis  of  risky  choice  in  behavioral  ecology  (see  Bateson  & Kacelnik,  1998; 
Kacelnik  & Bateson,  1996),  no  studies  have  explicitly  investigated  energy-budget  effects  in 
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humans.  It  is  possible  that  risky  choice  in  humans  may  show  the  same  shifts  from  risk  aversion  to 
risk  seeking  as  nonhumans  when  energy  budgets  are  manipulated. 

One  of  the  main  obstacles  to  studying  energy-budget  effects  in  humans  is  the  problem  of 
manipulating  food  intake.  The  effects  of  negative  energy-budgets  are  normally  studied  by 
restricting  food  access  and  presenting  food  reinforcers  during  experimental  sessions  at  a rate  that  is 
insufficient  to  meet  energy  requirements.  Even  if  human  participants  agreed  to  restrict  food  intake 
and  deprivation  levels  could  be  verified,  deprivation  levels  would  necessarily  be  far  above  those 
required  to  induce  negative-energy  budgets.  It  may  be  possible,  however,  to  create  experimental 
procedures  that  model  important  features  of  energy-budget  experiments  without  using  food  and/or 
life-death  choices.  That  is,  it  may  be  possible  to  use  a monetary  earnings  requirement  in  place  of 
an  energy  requirement.  The  reinforcement  contingencies  established  by  the  relationship  between 
choice  payoffs,  accumulated  earnings,  and  the  earnings  requirement  (an  “earnings”  budget)  could 
then  be  manipulated  to  simulate  positive  and  negative  energy  budgets.  Thus,  a second  aim  of  the 
present  research  was  to  develop  a procedure  to  assess  risk  sensitivity  in  humans  across 
manipulations  analogous  to  energy-budget  manipulations  conducted  with  nonhumans. 

Three  experiments  were  conducted.  The  first  two  experiments  investigated  whether  the 
results  of  previous  risky-choice  experiments  with  humans  would  generalize  to  situations  that  are 
more  similar  to  those  used  with  nonhumans.  Specifically,  human  subjects  were  given  repeated 
choices  between  fixed  and  variable  reinforcer  amounts  (Experiment  1)  and  fixed  and  variable 
reinforcer  delays  (Experiment  2)  across  several  variable  distribution  types.  The  choice  outcomes 
were  real  money.  These  experiments  were  designed  to  examine  whether  some  of  the  differences  in 
risky  choice  between  humans  and  nonhumans  are  due  to  species  differences  or  due  to 
methodological  differences  in  how  risky  choice  is  assessed.  The  third  experiment  investigated 
choice  in  humans  across  procedural  manipulations  designed  to  model  energy-budget  manipulations 
conducted  with  nonhumans.  This  experiment  was  designed  to  examine  whether  shifts  in  risk 
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sensitivity  shown  in  nonhumans  across  changes  in  energy  budget  would  also  occur  in  humans.  The 


results  of  these  experiments  may  contribute  to  the  analysis  of  human  risky  choice  in  the  laboratory 
and  in  the  field,  and  may  help  assess  the  generality  of  patterns  of  risky  choice  across  species. 
Furthermore,  because  the  experiments  combined  methods  and  procedures  from  several  different 
research  areas,  they  may  contribute  to  the  further  development  of  interdisciplinary  risky-choice 
research. 


EXPERIMENT  1 


FIXED  AND  VARIABLE  REINFORCER  AMOUNTS 
Overview 

Many  experiments  have  investigated  risk  sensitivity  by  presenting  subjects  with  choices 
between  a fixed  reinforcer  amount  that  is  delivered  with  certainty  and  a variable  reinforcer  amount 
that  is  composed  of  a probability  distribution  of  different  amount  values,  one  of  which  could  be 
zero.  For  example,  a risky-choice  experiment  with  nonhumans  may  present  subjects  with  choices 
between  a fixed  option  that  delivers  8 food  pellets  and  a variable  option  that  delivers  0 or  16  food 
pellets  with  equal  probability  (Hamm  & Shettleworth,  1987).  Alternatively,  an  experiment  may 
present  subjects  with  choices  between  a fixed  option  that  produces  3-s  access  to  food  with  certainty 
and  variable  option  that  produces  either  1-s  access  to  food  with  p=.75  or  9-s  access  to  food  with 
p=.25  (Staddon  & Innis,  1 966).  Choice  may  be  described  as  risk  averse  if  the  fixed  amount  is 
preferred  and  risk  prone  if  the  variable  amount  is  preferred. 

The  results  of  risky-choice  experiments  with  nonhumans  using  fixed  and  variable 
reinforcer  amounts  have  shown  all  types  of  risk  sensitivity.  Experiments  have  shown  that  choice  is 
risk  neutral  (Staddon  & Innis,  1966),  risk  prone  (Essock  & Reese,  1974;  Leventhal,  Morrell, 
Morgan,  & Perkins,  1959;  Young,  1981),  and  often  risk  averse  (Clements,  1990;  Hamm  & 
Shettleworth,  1987;  Logan,  1965;  Menlove,  Inden,  & Madden,  1979;  Reboreda  & Kacelnik,  1991; 
see  Kacelnik  & Bateson,  1996,  for  a review).  For  example,  Essock  and  Reese  (1974)  investigated 
choice  in  food-deprived  pigeons  by  presenting  subjects  with  choices  between  fixed  and  variable 
durations  of  food-access  time  having  the  same  mean  value.  Choice  of  four  of  five  subjects 
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was  risk  prone  across  conditions.  In  a similar  study,  Hamm  and  Shettleworth  (1987)  presented 
food-deprived  pigeons  with  choices  between  fixed  and  variable  numbers  of  food  pellets  when  both 
options  had  the  same  mean  value.  Choice  of  all  subjects  was  risk  averse. 

In  typical  risky-choice  experiments  with  humans,  subjects  are  presented  with  choices 
between  certain  and  variable  (usually  hypothetical)  amounts  of  money.  Although  these 
experiments  have  also  shown  all  types  of  risk  sensitivity,  the  predominant  finding  is  that  choice  is 
risk  averse,  at  least  when  choice  outcomes  are  gains  (e.g.,  Kahneman  & Tversky,  1979;  Kohn, 
Kohn,  & Staddon,  1992;  Schmitt  & Whitmeyer,  1990;  Schneider,  1992;  Schneider  & Lopes,  1986; 
Silberberg,  Murray,  Christensen,  & Asano,  1988;  Tversky  & Kahneman,  1992). 

For  example,  Kahneman  and  Tversky  (1979)  reported  that  when  95  subjects  were 
presented  with  the  following  choice  between  options  delivering  a hypothetical  amount  of  money: 

A:  (1,000,  p=.50)  or  B:  (500) 

84%  of  subjects  preferred  option  B,  the  certain  option.  Similarly,  Schneider  and  Lopes  (1986) 
presented  1,382  college  undergraduates  with  five  choices  between  certain  and  probabilistic 
hypothetical  amounts  of  money  in  a brief  questionnaire.  They  reported  that  certain  option  was 
preferred  in  the  majority  of  cases. 

Although  risky  choice  in  humans  appears  to  be  more  consistently  risk  averse  than  choice  in 
nonhumans,  there  are  many  procedural  differences  between  experiments  conducted  with  humans 
and  nonhumans  that  make  comparing  results  difficult.  Some  of  the  more  important  differences  are 
described  below.  In  the  present  experiment,  an  attempt  was  made  to  minimize  these  procedural 
differences  both  to  further  investigate  choice  in  humans  and  to  provide  a better  comparison  of 
human  and  nonhuman  risky  choice. 


7 


Verbal  Descriptions  of  Contingencies  versus  Exposure  to  Contingencies 

In  studies  with  nonhumans,  the  probabilities  of  receiving  each  reinforcer  amount  are 
actually  experienced  by  subjects  during  experimental  sessions.  Often,  subjects  are  exposed  to 
choice  outcomes  in  forced-choice  trials  before  the  actual  choice  trials  begin.  In  most  studies  with 
humans,  on  the  other  hand,  the  probabilities  of  receiving  each  amount  are  described  verbally  (e.g., 
Hershey  & Schoemaker,  1980;  Tversky  & Kahneman,  1992).  For  example,  a variable  amount 
used  in  a choice  problem  by  Tversky  and  Kahneman  (1981)  was  presented  as  the  verbal  statement: 
“80%  chance  to  win  $45.” 

No  studies  have  directly  compared  risky  choice  across  situations  in  which  amount 
probabilities  are  described  verbally  or  actually  experienced,  although  a few  studies  have  delivered 
reinforcers  probabilistically  during  experimental  sessions.  For  example,  Kohn  et  al.  (1992) 
presented  groups  of  college  undergraduates  with  repeated  choices  between  fixed  and  variable 
reinforcer  amounts,  reinforcer  delays,  and  rates  of  reinforcement,  using  a simple  computer  game. 
Subjects  participated  for  one  session  and  the  reinforcer  for  each  choice  was  the  appearance  of 
symbols  on  a computer  screen.  In  the  amount  conditions,  choice  outcomes  were  fixed  and  variable 
numbers  of  symbols.  The  variable  option  had  a bivalued  distribution.  In  one  experiment,  the 
amount  of  the  certain  option  was  manipulated  across  groups  and  was  either  equal  to,  greater  than, 
or  less  than  the  mean  value  of  the  variable  option.  For  all  three  groups,  choice  was  moderately  risk 
averse. 

Schmitt  and  Whitmeyer  (1990)  investigated  choice  in  five  humans  when  outcomes  were 
points  exchangeable  for  money.  One  option  delivered  a constant  number  of  points  and  the  second 
option  delivered  or  removed  points  with  an  equal  probability.  Subjects  were  exposed  to  a 
minimum  of  12,  50-min  sessions.  Three  subjects  showed  risk  aversion,  one  showed  indifference. 
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and  one  showed  risk  seeking.  Choice  of  four  of  the  five  subjects  became  more  risk  averse  across 
the  experiment. 

Finally,  in  a recent  study  by  Lane  and  Cherek  (1999),  12  subjects  were  presented  with 
repeated  choices  between  options  delivering  a small  amount  of  money  delivered  with  certainty  or  a 
high  probability,  and  larger  amount  of  money  delivered  with  a low  probability.  When  both  options 
had  approximately  equal  expected  values  (calculated  as  the  amount  multiplied  by  its  probability), 
choice  was  risk  averse. 

Overall,  these  findings  suggest  that  risk  aversion  may  be  common,  both  when  probabilities 
are  verbally  presented,  and  when  they  are  actually  experienced.  Because  choice  is  maintained  by 
different  types  of  contingencies  in  these  two  cases,  however,  it  is  likely  that  the  choice  patterns  are 
not  entirely  equivalent.  That  is,  in  the  first  case,  choices  are  influenced  by  past  experiences  with 
stated  probabilities,  but  in  the  second  case,  choices  are  influenced  by  contingencies  arranged  during 
the  experimental  setting.  That  patterns  of  risky  choice  maintained  by  these  different  contingencies 
may  not  be  equivalent  is  suggested  by  the  results  of  the  study  by  Schmitt  and  Whitmeyer  (1990), 
described  above.  In  their  study,  risk  sensitivity  changed  across  sessions,  indicating  that 
preferences  changed  with  continued  exposure  to  choice  outcomes. 

The  use  of  verbal  descriptions  of  contingencies  versus  actual  exposure  to  contingencies  (as 
well  as  the  use  of  one-shot  versus  repeated  choices,  described  below),  relates  to  a general  criticism 
concerning  human  operant  research  made  by  Baron,  Perone,  and  Galizio  (1991).  Most  studies 
with  humans  examine  behavior  across  very  short  periods  of  time  whereas  studies  with  nonhumans 
examine  behavior  until  performance  reaches  a steady  state.  It  is  difficult  to  directly  compare 
performance  of  humans  and  nonhumans  when  human  behavior  has  not  reached  stability  because 
early  in  an  experimental  condition  behavior  may  be  controlled  by  variables  other  than  the  current 
reinforcement  contingencies. 
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Because  the  primary  purpose  of  the  present  research  was  to  study  choice  in  humans  under 
conditions  that  resemble  those  used  with  nonhumans,  the  present  experiment  arranged  for  subjects 
to  experience  the  actual  choice  outcomes  (rather  than  merely  describing  them)  and  provided 
subjects  with  extensive  exposure  to  the  contingencies.  Furthermore,  conditions  remained  in  effect 
until  performance  reached  a steady  state. 

Repeated  versus  One-Shot  Choices 

Another  procedural  difference  between  research  conducted  with  humans  and  nonhumans 
concerns  the  amount  of  exposure  to  each  choice  outcome.  Most  experiments  with  humans  present 
each  choice  once  or  a few  times  only  whereas  most  experiments  with  nonhumans  present  the  same 
choices  repeatedly.  Several  studies  with  humans,  however,  have  found  differences  in  risk 
sensitivity  when  choice  is  compared  across  one-shot  (unique)  conditions,  in  which  a gamble  (i.e., 
probability  distribution)  is  presented  or  played  only  once,  and  repeated  conditions,  in  which  a 
gamble  is  presented  or  played  multiple  times.  For  example,  Keren  and  Wagenaar  (1987)  presented 
subjects  with  choices  between  pairs  of  identical  gambles  in  both  unique  and  repeated  conditions. 
Subjects  were  instructed  that  the  selected  gamble  would  be  played  once  in  the  unique  conditions 
and  ten  times  in  the  repeated  conditions.  During  unique  conditions,  the  certain  outcome  was 
preferred  to  a risky  outcome  despite  having  a lower  expected  value,  but  during  the  repeated 
conditions,  the  gamble  with  the  higher  expected  value  was  preferred 

Schmitt  and  Whitmeyer  (1990)  found  opposite  effects  of  presenting  repeated  versus  one- 
shot  choices.  At  the  end  of  their  experiment,  subjects  were  presented  with  a one-shot  choice 
between  the  certain  and  variable  option.  In  nearly  every  case,  choice  was  risk  prone  during  the 
one-shot  condition  whereas  choice  was  primarily  indifferent  or  risk  averse  during  the  experiment 
when  choices  were  presented  repeatedly. 
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Although  the  results  of  Schmitt  and  Whitmeyer  (1990)  experiment  conflict  with  those  of 
Keren  and  Wagenaar  (1987),  these  experiments  show  that  presenting  choices  under  unique  or 
repeated  conditions  can  produce  different  patterns  of  risk  sensitivity.  In  the  present  experiment,  to 
remain  consistent  with  the  procedures  used  to  study  choice  in  nonhumans,  subjects  were  presented 
with  repeated  choices. 


Nature  of  the  Consequence 

In  all  risky-choice  studies  with  nonhumans,  choices  produced  real  outcomes.  In  most 
risky-choice  studies  with  humans,  on  the  other  hand,  choices  produced  hypothetical  outcomes, 
usually  hypothetical  monetary  amounts  (e.g.  Currim  & Sarin,  1989;  Hershey  & Shoemaker,  1980; 
Kahneman  & Tversky,  1979;  Tversky  & Kahneman,  1992).  When  hypothetical  outcomes  are 
used,  it  is  usually  assumed  that  choices  are  similar  to  those  that  would  occur  if  real  outcomes  had 
been  used  (e.g.,  Kahneman  & Tversky,  1979).  Several  studies  with  humans,  however,  have 
explicitly  compared  risky  choice  when  outcomes  were  real  or  hypothetical,  and  have  shown  that 
choice  sometimes  varied  with  the  nature  of  the  consequence.  When  choice  outcomes  were  real, 
choice  has  been  shown  to  be  more  risk  averse  (Christensen,  Parker,  Silberberg,  & Hursh,  1998; 
Lafferty  & Higbee,  1974;  Slovic,  1969),  less  risk  averse  (Levin,  Chapman,  & Johnson,  1988),  or 
generally  similar  to  choice  when  outcomes  were  hypothetical  (Irwin,  McClelland,  & Schulze,  1992; 
Wiseman  & Levin,  1996).  To  minimize  the  procedural  differences  from  studies  with  nonhumans, 
the  present  experiment  used  real  monetary  outcomes. 

It  should  also  be  noted  that  previous  studies  using  real  monetary  outcomes  differed  in  an 
important  way  from  studies  with  nonhumans.  In  previous  experiments  with  humans  using  real 
outcomes,  with  very  few  exceptions  (e.g.  Lane  & Cherek,  1999;  Schmitt  & Whitmeyer,  1990), 
subjects  did  not  actually  experience  the  outcomes  of  choices  until  the  end  of  the  experiment.  That 
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is,  gambles  were  not  played  until  all  choices  had  been  made.  Thus,  there  was  no  opportunity  for 
consequences  to  influence  choice.  In  studies  with  nonhumans,  on  the  other  hand,  consequences  are 
delivered  repeatedly  throughout  the  experiment.  To  better  approximate  the  procedures  used  with 
nonhumans,  in  the  present  experiment  points  exchangeable  for  money  were  delivered  immediately 
following  each  reinforced  response. 

Form  of  the  Variable  Distribution 

Risky-choice  experiments  with  humans  and  nonhumans  also  differ  in  the  types  of  variable 
distributions  that  are  typically  used  to  assess  choice.  Experiments  with  humans  have  most  often 
used  two-outcome  (bivalued)  probability  distributions,  but  experiments  with  nonhumans  have 
investigated  choice  across  a much  wider  range  of  distribution  forms.  Furthermore,  several  studies 
with  nonhumans  have  shown  that  the  form  of  the  variable  distribution,  including  its  minimum 
value,  variance,  and  skew,  can  influence  preference. 

For  example,  Essock  and  Reese  (1974)  showed  that  when  pigeons  were  presented  with 
choices  between  fixed  and  variable  food-access  times,  choice  was  risk  prone  across  several 
different  distribution  forms,  but  choice  was  most  risk  prone  during  conditions  in  which  the  variable 
distribution  had  a high  minimum  value. 

Caraco  and  Lima  (1995)  presented  three,  dark-eyed  juncos  with  choices  between  a certain 
and  a variable  number  of  seeds.  They  reported  that  when  the  certain  option  was  pitted  against  one 
of  several  variable  distributions,  each  of  which  had  the  same  mean  value  but  different  standard 
deviations,  preference  for  the  certain  option  increased  as  the  standard  deviation  of  the  variable 
option  increased.  These  results  were  described  in  terms  of  a “trade-off’  between  mean  and 
variance.  Similar  results  have  been  shown  in  another  bird  species,  bananaquits  (Wunderle  & 
Cotto-Navarro,  1988).  Furthermore,  when  Caraco  and  Chasin  (1984)  presented  three,  white- 
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crowned  sparrows  with  choices  between  two  variable  distributions  having  the  same  mean  value  but 
different  skews,  the  positively  skewed  distribution  (mode  > median  > mean)  was  preferred  to  the 
negatively  skewed  distribution  (mean  < median  < mode). 

One  psychological  model  that  predicts  that  risky  choice  may  be  influenced  by  the  shape  of 
the  variable  amount  distribution  is  scalar  expectancy  theory,  or  SET  (Gibbon,  1977).  SET  was 
originally  proposed  as  a model  of  temporal  discrimination  but  has  been  extended  to  choice  between 
fixed  and  variable  reinforcer  delays  by  Gibbon,  Church,  Fairhurst,  and  Kacelnik  (1988). 
Predictions  of  SET  have  also  been  extended  to  choice  between  fixed  and  variable  reinforcer 
amounts  when  amounts  are  programmed  as  fixed  and  variable  durations  of  food  access  (Reboreda 
& Kacelnik,  1991).  According  to  Gibbon  et  al.,  SET  predicts  that  a variable  option  having  a 
bivalued,  rectangular,  or  positively  skewed  (exponential)  distribution  of  delays  will  be  preferred  to 
a fixed-delay  option  having  the  same  mean  value.  When  the  distribution  of  the  variable  option  is 
negatively  skewed  (reverse  exponential),  however,  the  fixed  and  variable  option  may  be  equally 
preferred.  Gibbon  et  al.  presented  data  from  a study  with  pigeons  consistent  with  these  predictions. 

One  of  the  few  studies  with  humans  examining  the  effects  on  choice  of  the  form  of  the 
variable  distribution  was  conducted  by  Lopes  (1984).  Subjects  were  presented  with  choices 
between  pairs  of  lotteries,  or  a certain  outcome  and  a lottery  having  the  same  mean  value 
(hypothetical  $100).  A variety  of  distributions  was  used  including  normal,  rectangular,  bivalued, 
and  skewed.  Lopes  ranked  each  distribution  according  to  a measure  of  variability  called  the 
lottery’s  inequality.  Distributions  were  presented  as  bar  graphs.  Subjects  were  classified 
according  to  their  preferences  between  certain  and  variable  gambles.  Overall,  choice  in  46%  of 
subjects  was  risk-averse,  choice  in  34%  of  the  subjects  was  indifferent,  and  choice  in  20%  of  the 
subjects  was  risk  prone.  For  subjects  whose  choices  were  risk  averse,  preference  for  a lottery 
decreased  as  its  inequality  increased,  with  lotteries  having  the  lowest  inequality  being  most  strongly 
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preferred  (i.e.,  gambles  with  a negative  skew  and  a low  probability  of  receiving  $0).  For  subjects 
whose  choices  were  risk  prone,  the  opposite  pattern  was  observed.  In  a second  study,  subjects 
were  also  presented  with  choices  between  positively  and  negatively  skewed  distributions  with  equal 
standard  deviations.  The  minimum  value  of  the  negatively  skewed  distribution  was  $0  but  the 
minimum  value  of  the  positively  skewed  distribution  was  above  $0.  For  subjects  whose  choices 
were  risk  averse,  the  positively  skewed  distribution  was  preferred. 

Lopes’  (1984)  research  shows  that  choice  of  humans,  like  choice  of  nonhumans,  is 
sensitive  to  distributional  characteristics.  The  procedures  used  by  Lopes,  however,  were  very 
different  from  those  used  in  studies  with  nonhumans:  Subjects  were  given  few  choice  trials  and 
amount  probabilities  were  described  verbally  (i.e.,  in  graph  form)  rather  than  being  actually 
delivered. 

The  present  study  sought  to  investigate  further  the  effects  on  risky  choice  of  the  form  of 
the  variable  distribution  by  presenting  subjects  with  repeated  choices  between  fixed  and  variable 
amounts  when  the  amount  probability  distributions  were  actually  experienced. 

Specific  Rationale  for  Experiment  1 

The  many  procedural  differences  between  risky-choice  experiments  with  humans  and 
nonhumans  make  comparing  patterns  of  risk  sensitivity  difficult.  Thus,  the  aim  of  Experiment  1 
was  to  determine  whether  patterns  of  risk  sensitivity  shown  in  humans  during  typical  risky-choice 
experiments  would  generalize  to  choice  contexts  that  more  closely  approximate  those  used  with 
nonhumans. 

As  mentioned  previously,  one  notable  procedural  difference  between  experiments  with 
humans  and  nonhumans  is  that  experiments  with  nonhumans  arrange  for  subjects  to  experience 
choice  contingencies  whereas  most  studies  with  humans  only  present  subjects  with  verbal 
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descriptions  of  choice  contingencies.  The  present  experiment  attempted  to  control  for  these 
differences  by  arranging  for  human  subjects  to  experience  choice  outcomes  directly.  This  was 
accomplished  in  two  ways.  First,  the  different  reinforcer  amounts  were  actually  delivered  during 
experimental  sessions  according  to  their  programmed  probabilities.  For  example,  when  the 
variable  distribution  was  bivalued,  the  variable  option  added  either  25  or  75  points  with  equal 
probability  to  a counter  and  the  fixed  option  always  added  50  points  to  the  counter.  Second,  half 
of  the  trials  of  each  session  were  forced-choice  trials  to  guarantee  exposure  to  both  the  fixed  and 
variable  outcomes  before  the  choice  trials  were  presented. 

A second  procedural  difference  concerns  the  degree  of  exposure  to  contingencies. 
Experiments  with  nonhumans  present  subjects  with  repeated  choices  between  fixed  and  variable 
amounts  but  typical  experiments  with  humans  present  subjects  with  few  choices.  Thus,  in  the 
present  experiment  subjects  were  given  long-term  exposure  to  choice  contingencies.  Specifically, 
each  experimental  session  consisted  of  30  choice  trials  and  conditions  were  in  effect  for  a minimum 
of  six  sessions  and  until  performance  stabilized.  The  total  number  of  sessions  each  subject 
experienced  ranged  from  43  to  74. 

Third,  choice  outcomes  in  risky-choice  experiments  with  nonhumans  are  nearly  always 
real,  whereas  choice  outcomes  in  experiments  with  humans  are  most  often  hypothetical.  To  more 
closely  approximate  the  procedures  used  with  nonhumans,  the  present  experiment  used  points 
exchangeable  for  real  money  as  consequences. 

Finally,  experiments  with  humans  have  investigated  choice  across  a variety  of  different 
distribution  forms,  but  most  experiments  with  humans  have  used  only  bivalued  distributions.  In 
the  present  experiment,  choice  was  assessed  across  four  different  probability  distributions.  The 
form  of  each  distribution,  shown  in  Figure  1,  was  either  bivalued,  rectangular,  (approximately) 
normal,  or  reverse  exponential  (negatively  skewed).  Also  shown  in  Figure  1 is  the  standard 
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Fig.  1.  Probability  distributions  used  in  Experiment  1.  SD  indicates  the  standard 
deviation  and  N indicates  the  number  of  elements  in  each  distribution. 


deviation  and  number  of  elements  in  each  distribution.  All  distributions  had  the  same  mean  value 
and,  with  the  exception  of  the  exponential  distribution,  the  same  minimum  value.  If  choice  of 
humans  is  consistent  with  predictions  of  choice  models  such  as  SET,  then  choice  would  be  more 
risk  averse  when  the  certain  option  was  paired  with  the  bivalued,  rectangular,  or  normal 
distributions,  than  when  paired  with  the  reverse-exponential  distribution.  If  risky  choice  is 
influenced  by  the  degree  of  variability,  then  choice  may  be  more  risk  averse  when  the  variable 
distribution  has  a high  standard  deviation  (i.e.,  when  the  distribution  is  bivalued)  than  when  it  has  a 
low  standard  deviation  (i.e.,  when  the  distribution  is  approximately  normal). 
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Method 

Subjects 

The  subjects  were  three  adult  humans  recruited  via  an  advertisement  in  a local  university 
newspaper.  Subject  21 1 was  a 20  year-old  male  and  Subjects  212  and  21 8 were  19  and  20  year- 
old  females,  respectively.  None  had  any  previous  experience  with  behavioral  research.  At  the 
termination  of  the  experiment,  subjects  were  mailed  a check  for  money  earned  during  each 
experimental  session  (approximately  $3.00  per  session)  plus  $1 .50  for  each  completed  session. 
Overall,  subjects  earned  approximately  $6.00  per  hour. 

Apparatus 

Subjects  were  seated  in  a cubicle  measuring  2.21  m high,  1 .21  m wide,  and  1.25  m deep, 
facing  a white,  aluminum  panel  74  cm  high  and  44.5  cm  wide.  The  upper  portion  of  the  panel 
contained  3 rows  of  12  bulbs,  with  each  row  spaced  8 cm  apart.  The  first  and  last  bulb  of  each 
row  were  covered  with  white  translucent  caps;  the  1 0 inner  bulbs  of  each  row  were  covered  with 
red  translucent  caps.  Three  2.5  cm  response  keys,  positioned  8.2  cm  from  one  another,  were 
mounted  8.2  cm  below  the  lowest  row  at  approximately  eye  level.  A force  of  approximately  0.6  N 
was  required  to  operate  the  response  keys.  A 4-digit  2.5  cm  by  5 cm  electromechanical  counter 
was  mounted  19.5  cm  below  the  left  response  key.  The  counter  was  centered  in  an  8.8  cm  by  7.7 
cm  opening  on  the  aluminum  panel  covered  with  Plexiglas.  An  overhead  light  provided  diffuse 
illumination  and  a ventilation  fan  helped  mask  extraneous  sound. 

Several  changes  were  made  to  the  apparatus  before  Subject  218  began  the  experiment. 
First,  the  lower  section  (23.5  cm)  of  the  aluminum  panel  was  removed  and  replaced  with  a white, 
wooden  panel.  A 20  cm  by  15  cm  opening  for  a television  monitor  (not  used  in  the  present 
experiment)  was  positioned  4 cm  from  the  left  side  of  the  panel  and  9.8  cm  below  the  response 
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keys.  A predetermining  counter  (not  used  in  the  present  experiment)  was  mounted  7 cm  to  the  right 
of  the  monitor.  The  4 digit  electromechanical  counter  was  re-positioned  beneath  the  right  key,  3 
cm  to  the  right  of  the  predetermining  counter  and  1 5 cm  above  the  bottom  of  the  panel.  Both 
counters  were  mounted  directly  on  the  wooden  panel. 

Procedure 

Each  session  consisted  of  60  trials  (30  forced-choice  trials  and  then  30  choice  trials)  in 
which  choices  could  produce  either  a fixed  or  variable  number  of  points.  During  choice  trials,  the 
left  and  right  keys  were  illuminated  and  flashed  according  to  a 0.25  s on-off  cycle.  The  fixed  and 
variable  alternatives  were  correlated  with  yellow  and  green  keylights,  respectively.  The  side 
position  of  the  fixed  and  variable  alternative  was  randomly  determined  each  trial.  During  the 
choice  period,  five  consecutive  responses  on  one  of  the  keys  stopped  the  keylight  from  flashing  (the 
keylight  became  continuously  illuminated),  disabled  the  alternative  schedule,  and  initiated  a 10-s 
delay.  The  first  response  after  1 0 s had  elapsed  darkened  the  response  key  and,  depending  on  the 
schedule  type,  produced  a fixed  number  of  points  (50)  or  a variable  number  of  points  (averaging 
50).  During  the  delay  period,  a 5-s  delay  was  programmed  between  responses  on  dark  keys  and 
point  presentations.  For  Subjects  21 1 and  212,  during  a pre-training  period  lasting  16  and  5 
sessions,  respectively,  only  a single  response  was  required  during  the  choice  period  and  one 
additional  response  was  required  to  produce  points. 

Forced-choice  trials  were  similar  to  choice  trials  except  that  only  one  side  key  was 
illuminated.  The  schedule  type  (fixed  or  variable)  was  randomly  determined  each  trial  with  the 
following  restrictions:  (a)  Neither  schedule  type  could  occur  consecutively  on  more  than  three 
trials,  and  (b)  each  schedule  type  occurred  on  half  of  the  trials.  To  hold  the  rate  of  trial  onset 
constant,  trials  were  presented  every  45  s.  Because  the  time  required  to  deliver  points  varied  with 
the  point  amount,  the  actual  inter-trial  interval  (IT1)  could  vary  from  trial  to  trial. 
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Prior  to  the  first  experimental  session,  the  following  instructions  were  presented  and  read 
to  each  subject: 

Every  10  points  displayed  on  the  counter  is  worth  $0.01 . For 
example,  if  the  counter  shows  500  points  you  have  earned  $0.50. 

You  can  earn  points  by  pressing  the  response  keys  when  lit. 

Please  remain  seated.  You  will  be  informed  when  the  session  is 
over. 

Choice  was  assessed  across  four  distribution  types:  bivalued,  (approximately)  normal, 
reverse  exponential,  and  rectangular.  Table  1 shows  the  sequence  and  number  of  sessions  per 
condition  for  each  subject.  The  order  of  exposure  to  each  distribution  varied  across  subjects  and  in 
most  cases  each  distribution  was  presented  twice.  Conditions  were  changed  after  a minimum  of  six 
sessions  and  when  the  number  of  choices  for  the  fixed  option  was  stable  across  a five-session 
block,  as  determined  by  visual  inspection.  Sessions  were  conducted  at  approximately  the  same 
hour,  Monday  through  Friday.  Typically,  two  sessions  were  conducted  daily  and  sessions  were 
separated  by  a minimum  of  1 0 min. 


Table  1 


Sequence  and  number  of  sessions  per  condition  for  each  subject  in  Experiment  1. 


Condition 

Subject 

211 

212 

218 

Bivalued 

2(9) 

3(6) 

4(6) 

5(7) 

8(11) 

Exponential 

4(6) 

1(18) 

2(12) 

7(7) 

6(6) 

5(6) 

9(6) 

Normal 

1(5) 

2(9) 

3(6) 

6(8) 

5(6) 

9(6) 

Rectangular 

3(12) 

4(6) 

1(7) 

8(7) 

7(6) 

6(6) 
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Results 

Figure  2 shows  the  mean  number  of  choices  for  the  fixed  option  per  session  across  the 
final  five  sessions  of  each  condition.  Filled  bars  show  the  results  from  the  initial  exposure  to  each 
distribution  and  open  bars  show  the  replications.  Vertical  lines  show  the  standard  deviations  of  the 
session  values.  The  horizontal  line  indicates  the  indifference  point  between  the  two  options. 

Choice  was  defined  as  risk  averse  if  the  number  of  choices  for  the  fixed  option  was  above  this  line, 
and  risk  prone  if  the  number  of  choices  was  below  it. 

Choice  was  risk  averse  in  most  conditions  and  there  was  no  systematic  effect  on  choice  of 
the  form  of  the  distribution.  For  Subject  211,  choice  was  strongly  risk  averse  during  the  first 
condition  (normal  distribution).  Across  subsequent  conditions,  the  mean  number  of  choices  for  the 
fixed  option  decreased  for  this  subject,  but  remained  consistently  above  15  in  all  conditions.  For 
Subject  212,  choice  was  only  slightly  risk  averse  during  the  first  condition  of  the  experiment 
(exponential  distribution),  but  was  strongly  risk  averse  during  the  remaining  conditions.  Similarly, 
for  Subject  218  choice  was  slightly  risk  prone  during  the  first  condition  (rectangular  distribution) 
and  slightly  risk  averse  during  the  second  condition  of  the  experiment  (exponential  distribution), 
but  was  strongly  risk  averse  during  the  remaining  conditions. 

The  right  and  left  panels  of  Figure  3 show  for  each  subject  the  mean  obtained  point 
amounts  per  trial  on  the  variable  option  during  forced-choice  and  choice  trials,  respectively. 
Because  differences  between  the  first  and  second  exposures  to  each  distribution  were  generally 
small,  results  from  both  exposures  have  been  combined.  The  vertical  lines  show  the  standard 
deviations  of  the  values  from  both  exposures.  During  forced-choice  trials,  the  average  number  of 
obtained  points  on  the  variable  option  was  very  close  to  the  value  of  the  fixed  amount  (50  points) 
across  all  distribution  types.  During  choice  trials,  the  average  number  of  obtained  points  was  also 
near  50  in  most  cases  but,  as  a result  of  the  fewer  number  of  variable  choices,  the  values  showed 
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Fig.  2.  Mean  number  of  choices  for  the  fixed  option  per  session  across  the  final 
five  sessions  of  each  exposure  to  the  rectangular  (RTG),  exponential  (EXP), 
bivalued  (BV),  and  normal  (NM)  distributions  for  each  subject  in  Experiment  1 . 
The  filled  bars  show  values  from  the  initial  exposures  to  each  condition  and  open 
bars  show  values  from  the  replications.  The  horizontal  line  shows  the  indifference 
point  between  the  two  options.  Vertical  lines  show  standard  deviations. 
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Fig.  3.  Mean  obtained  point  amounts  on  the  variable  option  per  trial  during 
exposures  to  the  rectangular  (RTG),  exponential  (EXP),  bivalued  (BV),  and 
normal  (NM)  distributions  for  each  subject  during  forced-choice  trials  (left  panel) 
and  choice  trials  (right  panel).  Values  from  initial  exposures  to  each  condition  and 
replications  have  been  combined.  The  horizontal  line  shows  the  point  amount  of 
the  fixed  option  per  trial  (50  points).  Vertical  lines  show  standard  deviations. 


22 


somewhat  more  variability.  For  Subject  212,  the  obtained  point  amounts  were  zero  during  both 
exposures  to  the  rectangular  distribution  due  to  an  exclusive  preference  for  the  fixed  option. 

Discussion 

When  three  subjects  were  given  repeated  choices  between  fixed  and  variable  amounts  of 
points  exchangeable  for  money  across  a prolonged  exposure  to  experimental  contingencies,  choice 
of  all  subjects  was  risk  averse.  It  is  unlikely  that  these  results  can  be  attributed  to  differences  in 
average  reinforcement  rates  between  the  fixed  and  variable  options,  because  the  obtained  average 
point  amounts  were  generally  similar  on  both  the  fixed  and  variable  options. 

The  overall  pattern  of  risk  sensitivity  is  thus  consistent  with  previous  research  with 
humans  using  few  choice  trials  and  hypothetical  amounts  of  money  (Kahneman  & Tversky,  1979; 
Schmitt  & Whitmeyer,  1990).  That  choice  patterns  varied  across  initial  exposures  to  conditions 
and  replicated  conditions  suggests,  however,  that  extensive  exposure  to  choice  outcomes  may 
produce  different  patterns  of  risk-sensitivity  than  single-shot  choice  procedures. 

In  contrast  to  the  results  of  previous  studies  with  nonhumans  and  the  predictions  of  several 
choice  models,  the  form  of  the  variable  amount  distribution  had  no  effect  on  preference.  For  two 
subjects  (Subjects  212  and  218),  it  is  likely  that  the  strong  risk  aversion  obscured  any  effect  that 
the  different  distribution  forms  may  have  had  on  choice.  For  example,  Lopes  (1984)  showed  that 
subjects  who  showed  a preference  for  a fixed  option  over  several  different  variable  distributions 
showed  a preference  for  less  variable  distributions  over  more  variable  distributions  when  both 
choice  outcomes  were  variable.  It  is  possible,  therefore,  that  if  both  choice  options  in  the  present 
experiment  were  variable  amounts,  choice  of  these  two  subjects  may  have  showed  greater 
sensitivity  to  distributional  characteristics.  For  Subject  211,  however,  choice  was  much  less  risk 
averse  than  in  the  other  two  subjects,  but  choice  did  not  vary  as  a function  of  the  distribution  form. 
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Overall,  the  results  of  the  present  experiment  suggest  that  the  patterns  of  risk-sensitivity 
shown  in  previous  studies  with  humans  generalize  to  situations  that  are  more  similar  to  those  used 
to  study  choice  in  nonhumans.  These  results  also  suggest  that  some  of  the  procedural  differences 
between  experiments  with  humans  and  nonhumans,  including  the  presentation  of  probabilities, 
repeated  versus  one-shot  choices,  real  versus  hypothetical  outcomes,  and  the  form  of  the  variable 
distribution,  cannot  account  for  the  more  frequent  reports  of  risk  seeking  in  studies  with 
nonhumans  than  in  studies  with  humans. 


EXPERIMENT  2 


FIXED  AND  VARIABLE  DELAYS  TO  REINFORCEMENT 

Overview 

Analyses  of  risky  choice  have  traditionally  focused  on  preferences  between  fixed  and 
variable  reinforcer  amounts,  but  amount  is  only  one  dimension  along  which  variability  can  occur. 
Variability  may  also  occur  in  the  delay  to  reinforcement.  Experiments  investigating  choice 
between  fixed  and  variable  delayed  reinforcers  have  not  usually  been  conceptualized  in  terms  of 
risk,  most  likely  because  the  variables  controlling  preferences  for  amounts  and  delays  are  assumed 
to  be  different  (but  see  Rachlin,  Logue,  Gibbon  & Frankel,  1986;  Reboreda  & Kacelnik,  1991).  In 
both  cases,  however,  choice  in  humans  and  nonhumans  has  been  shown  to  be  sensitive  to 
variability  (i.e.,  risk  sensitive). 

In  choice  experiments  using  delays,  subjects  are  typically  presented  with  choices  between  a 
fixed  option,  in  which  the  reinforcer  delay  is  constant,  and  a variable  option,  in  which  different 
reinforcer  delays  occur  probabilistically.  Preference  for  the  fixed-delay  option  may  be  described  as 
risk  averse,  and  preference  for  the  variable-delay  option  may  be  described  as  risk  prone. 

Although  choice  in  nonhumans  appears  to  be  less  consistently  risk  averse  than  choice  in 
humans  when  outcomes  are  amounts,  differences  in  risk  sensitivity  between  nonhumans  and 
humans  appear  to  be  much  more  extreme  when  outcomes  are  delays.  Experiments  with 
nonhumans  have  repeatedly  shown  that  choice  for  reinforcer  delays  is  strongly  risk  prone  (Bateson 
& Kacelnik,  1995;  Cicerone,  1976;  Davison,  1969,  1972;  Frankel  & Vom  Saal,  1976;  Herrnstein, 
1964;  Killeen,  1968;  Mazur,  1984;  Pubols,  1962).  Nonhumans  have  also  shown  a preference  for 
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variable-ratio  (VR)  over  fixed-ratio  (FR)  schedules  of  reinforcement,  and  this  finding  is  usually 
interpreted  in  terms  of  a preference  for  variable  delays  to  reinforcement  (Fantino,  1967;  Rider, 
1983;  Sherman  & Thomas,  1968).  To  the  author’s  knowledge,  however,  only  two  studies  have 
investigated  risky  choice  in  humans  with  fixed  and  variable  reinforcer  delays,  and  the  results  of 
both  of  these  studies  were  inconsistent  with  results  of  studies  with  nonhumans. 

Weiner  (1966)  investigated  choice  in  humans  between  two  ratio  schedules  of  point 
delivery.  In  one  condition,  the  two  schedules  were  a FR  40  and  a VR  40  schedule.  The  elements 
of  the  variable  schedule  varied  randomly  from  1 to  80.  Weiner  found  no  strong  preference  for 
either  schedule.  These  results  are  inconsistent  with  previous  research  with  nonhumans  that  has 
shown  that  VR  schedules  are  highly  preferred  to  FR  schedules  of  the  same  mean  value  (e.g., 
Fantino,  1967). 

In  some  conditions  of  a study  described  earlier,  Kohn  et  al.  (1992)  presented  subjects  with 
repeated  choices  between  fixed  and  variable  delays  to  the  presentation  of  symbols  on  a computer 
screen.  A sequence  of  four  experiments  was  conducted.  The  delay  schedules  were  fixed-time  (FT) 
and  variable-time  (VT)  schedules  in  which  symbols  were  delivered  after  fixed  and  variable  delays 
from  the  choice;  no  additional  response  was  required  to  produce  symbols  once  the  delay  timed  out. 
In  two  experiments,  the  variable  option  was  a VT  5-s  schedule  composed  of  two  intervals,  1 s and 
9 s (p=.5).  Across  groups  of  subjects,  the  FT  schedule  either  equaled,  was  greater,  or  was  lower 
than  the  arithmetic  mean  of  the  VT  schedule.  Choice  was  risk  prone  when  the  FT  schedule  value 
was  greater  than  the  mean  value  of  the  VT  schedule,  but  was  risk  averse  when  the  FT  schedule 
value  was  equal  to  or  greater  than  the  mean  value  of  the  VT  schedule.  In  another  experiment,  the 
intervals  of  the  VT  schedule  were  either  1 and  19  s (Long-delay  group)  or  1 and  3 s (Short-delay 
group).  The  value  of  the  FT  schedule  equaled  the  mean  value  of  the  VT  schedule.  In  both  groups, 


choice  was  risk  averse. 
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The  results  of  the  three  experiments  by  Kohn  et  al.  (1992)  contrast  with  the  results  of 
research  with  nonhumans  in  which  variable  delays  to  reinforcement  were  strongly  preferred  to 
fixed  delays  having  the  same  mean  value  (e.g.  Cicerone,  1976;  Pubols,  1962).  In  only  one  of  their 
experiments  was  choice  moderately  risk  prone.  In  that  experiment,  no  distinct  stimulus  was 
correlated  with  the  choice,  outcome,  or  ITI  portion  of  a trial. 

Although  risky  choice  in  nonhumans  appears  quite  different  than  risky  choice  in  humans 
when  outcomes  are  reinforcer  delays,  it  is  difficult  to  directly  compare  these  results.  The  main 
problem  is  that  there  are  very  few  studies  with  humans  on  which  to  base  a comparison.  For 
example,  many  risky  choice  studies  with  nonhumans  have  used  VI  and  FI  reinforcement  schedules 
(e.g.,  Hermstein,  1964;  Killeen,  1968),  but  studies  with  humans  have  only  used  VR  and  FR  and 
VT  and  FT  schedules.  In  addition,  many  studies  with  nonhumans  have  shown  that  the  form  of  the 
variable  delay  distribution  can  have  large  effects  on  risk  sensitivity.  No  studies  with  humans, 
however,  have  yet  investigated  the  effects  on  choice  of  distributional  characteristics. 

Form  of  the  Variable  Distribution 

Most  studies  that  have  shown  that  risky  choice  is  influenced  by  the  form  of  the  variable 
delay  distribution  were  not  designed  to  study  risky  choice  per  se.  Rather,  they  were  designed  to 
identify  the  averaging  principle  that  best  characterizes  the  reinforcement  rate  or,  more  generally, 
the  value  of  a variable  reinforcement  schedule.  This  averaging  principle  is  then  used  to  predict 
preference  between  pairs  of  reinforcement  schedules  with  delayed  outcomes.  A variety  of 
averaging  principles  have  been  proposed,  but  all  of  them  assume  that  the  reinforcing  effectiveness 
of  a choice  outcome  decreases  as  a function  of  its  delay.  That  is,  reinforcers  that  occur  with  a 
short  delay  are  assumed  to  have  a stronger  effect  on  choice  than  reinforcers  that  occur  with  a 


longer  delay. 
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Killeen  (1968),  for  example,  presented  pigeons  with  choices  between  several  different  FI 
and  VI  reinforcement  schedules  to  determine  the  FI  schedule  that  was  equally  preferred  to  the  VI 
schedule.  He  reported  that  the  harmonic  mean  of  the  VI  distribution  provided  a good  estimate  of 
the  FI  schedule  value  at  which  choice  was  indifferent  between  the  two  schedules.  Harmonic 
means,  unlike  arithmetic  means,  differentially  weight  the  short  delay  values  of  a distribution.  To 
test  this  model  further,  Killeen  presented  pigeons  with  choices  between  two  VI  schedules.  The 
distributions  of  the  two  schedules  were  constructed  so  that  one  schedule  had  a higher  arithmetic 
mean  but  a lower  harmonic  mean  than  the  other.  All  subjects  showed  a preference  for  the  VI 
schedule  with  the  lower  harmonic  mean. 

Averaging  principles  comparable  to  the  harmonic  mean  have  also  been  proposed  by 
Bateson  and  Kacelnik  (1995,  1996)  and  Mazur  (1994,  1996).  Bateson  and  Kacelnik  showed  that 
an  averaging  principle  based  upon  the  short-term  rate  of  reinforcement  (gain  per  interval  divided  by 
the  interval),  called  the  expectation  of  ratios  (EoR),  provided  a better  description  of  preference  than 
an  averaging  principle  based  upon  the  mean  rate  of  reinforcement  (total  gain  divided  by  total  time). 
Similarly,  Mazur  showed  that  a hyperbolic  discounting  function  provided  a good  estimate  of  the 
fixed  schedule  values  that  were  equally  preferred  by  pigeons  to  several  different  variable  schedules. 
According  to  Mazur’s  model,  the  value  of  a variable  schedule  is  determined  by  the  sum  of  the 
reciprocals  of  each  delay  interval,  with  each  delay  weighted  by  its  probability  of  occurrence. 

All  of  the  averaging  principles  described  above  predict  that  the  short  delays  of  a variable 
distribution  are  given  more  weight  than  the  long  delays  in  calculations  of  reinforcement  rate  or 
value.  Thus,  these  models  would  predict  that  the  relative  frequency  of  short  and  long  delay  values 
in  a variable  distribution  (i.e.,  the  form  of  the  variable  distribution)  will  influence  risk  sensitivity. 

Scalar  expectancy  theory,  or  SET,  also  predicts  that  choice  should  be  risk  prone  for 
reinforcer  delays  and  that  the  form  of  the  variable  delay  distribution  influences  choice  (Gibbon  et 
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al.,  1988).  As  described  above,  SET  predicts  that  a variable  option  will  be  preferred  to  a fixed 
option  when  the  variable  distribution  is  rectangular,  bivalued,  or  forward  exponential  (positively 
skewed).  Choice  should  be  risk  neutral,  however,  when  the  variable  distribution  has  high 
proportion  of  long  delay  elements,  such  as  when  the  distribution  is  reverse  exponential  (negatively 
skewed).  Support  for  these  predictions  was  provided  by  two  experiments  with  pigeons  using  VI 
schedules  with  forward-exponential  and  reverse-exponential  distributions  (Gibbon  et  al.,  1988). 
These  experiments  showed  that  the  VI  schedule  was  preferred  to  a constant-delay  option  when  the 
form  of  the  variable  distribution  was  forward  exponential,  but  that  the  VI  schedule  was  equally 
preferred  to  the  constant-delay  option  when  form  of  the  distribution  was  reverse  exponential. 

Two  other  distributional  characteristics  that  have  been  investigated  with  nonhumans  are 
the  number  of  delay  intervals  and  the  value  of  the  smallest  delay  interval  in  the  variable 
distribution.  For  example,  Davison  (1972)  presented  pigeons  with  choices  between  FI  and  VI 
reinforcement  schedules  with  the  same  mean  value.  Across  conditions,  the  number  of  intervals  in 
the  VI  schedule  was  manipulated  while  the  minimum  (and  maximum)  interval  was  held  constant. 
Davison  showed  that  preference  remained  constant  across  all  conditions. 

Results  of  several  studies  have  suggested  that  the  smallest  value  of  a variable  schedule 
influences  choice  (Ahearn  & Hineline,  1992;  Cicerone,  1976;  Duncan  & Fantino,  1970;  Fantino, 
1967).  Ahearn  and  Hineline,  for  example,  presented  pigeons  with  choices  between  mixed-ratio 
(MR)  schedules  composed  of  two  ratio  values  and  FR  schedules  having  the  same  or  lower 
arithmetic  mean  value.  The  results  showed  that  a MR  60.5  schedule  composed  of  ratio  values  of  1 
and  120  was  preferred  to  a FR  15  schedule,  but  that  the  FR  15  schedule  was  preferred  to  a MR 
60.5  schedule  composed  or  ratio  values  of  15  and  105.  In  a similar  study,  Fantino  (1967)  showed 
that  preference  of  pigeons  for  MR  50  schedules  paired  with  FR  50  schedules  increased  as  the 


lowest  value  of  the  MR  schedule  increased  from  1 to  25. 
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These  studies  show  that  distributional  characteristics,  including  the  form  of  the  variable 
distribution  and  the  minimum  value  of  a variable  distribution,  can  have  large  effects  on  risky 
choice  for  delayed  outcomes  in  nonhumans.  The  effects  of  distributional  characteristics  on  risky 
choice  in  humans,  however,  have  not  been  studied  directly.  In  both  the  Weiner  (1966)  and  the 
Kohn  et  al.  (1996)  experiments,  only  one  variable  distribution  was  used  (rectangular  and  bivalued, 
respectively).  Thus,  a primary  goal  of  the  present  experiment  was  to  investigate  risky-choice  in 
humans  across  several  different  variable  distributions.  These  manipulations  would  not  only 
determine  whether  choice  of  humans  is  influenced  by  distributional  characteristics,  but  would  also 
provide  a better  basis  for  comparing  risky  choice  in  humans  and  nonhumans  for  delayed  outcomes. 

Probability  Discounting  of  Monetary  Reinforcers 
One  interpretation  of  the  risk  indifference  and  risk  aversion  shown  in  the  experiments  by 
Weiner  (1966)  and  Kohn  et  al.  (1996)  is  that  the  reinforcers  (points  and  symbol  presentations) 
were  not  discounted  by  their  delay.  Research  with  probabilistic  monetary  reinforcers  has 
suggested,  however,  that  choice  of  humans  does  indeed  show  delay  discounting. 

Rachlin  et  al.  (1986)  proposed  that  probabilistic  reinforcers  are  functionally  equivalent  to 
fixed  reinforcers  that  are  delayed.  Probabilistic  reinforcers  are  similar  to  delayed  reinforcers 
because  they  are  delivered  after  a variable  number  of  choices.  For  example,  if  choice  trials  are 
separated  by  60-s  ITIs,  a reinforcer  that  occurs  with  p=.25  will  occur  on  average  after  a 1 80  s 
delay  (i.e.,  after  3 ITIs).  Delay  discounting  can  thus  explain  risk  aversion  for  certain  and 
probabilistic  reinforcer  amounts.  That  is,  because  probabilistic  reinforcers  are  discounted,  when 
subjects  are  presented  with  choices  between  certain  and  probabilistic  reinforcers,  the  certain 
reinforcer  is  preferred.  (It  should  be  noted  that  this  interpretation  of  risk  aversion  applies  only  for 
amount  distributions  with  no-gain  or  zero  amount  values). 
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Results  of  a number  of  experiments  with  humans  (Rachlin,  Castrogiovanni,  & Cross, 

1987;  Rachlin  et  al.  1986;  Rachlin,  Raineri,  & Cross,  1991;  Rachlin  & Siegel,  1994)  and 
nonhumans  (King,  Logue,  & Gleiser,  1992;  Mazur,  1989,  1991;  Mazur  & Romano,  1992)  have 
generally  supported  this  assumption  (but  see  Christensen  et  al.,  1998;  Green,  Myerson,  & 
Ostaszewski,  1999  for  one  inconsistency  between  delay  and  probability  discounting).  These 
studies  thus  suggest  that  when  outcomes  are  monetary  reinforcers,  choice  of  humans,  like  choice  of 
nonhumans,  may  show  delay  discounting. 

Specific  Rationale  for  Experiment  2 

A substantial  amount  of  research  with  nonhumans  has  shown  that  variable  delays  to 
reinforcement  are  strongly  preferred  to  fixed  delays,  yet  very  few  studies  have  been  conducted  with 
humans.  This  difference  may  account  for  the  more  frequent  reports  of  risk  proneness  in  risky- 
choice  experiments  with  nonhumans.  Thus,  the  present  research  sought  to  investigate  further  risky 
choice  in  humans  with  delayed  reinforcers.  Specifically,  subjects  were  presented  with  repeated 
choices  between  FI  and  VI  schedules  of  reinforcement  with  approximately  equal  arithmetic  mean 
values  (30  s).  Although  risky  choice  in  nonhumans  has  been  frequently  assessed  with  FI  and  VI 
schedules,  no  experiments  have  investigated  risky  choice  in  humans  with  these  schedule  types.  The 
mean  value  of  the  FI  and  VI  schedules  was  selected  to  approximate  the  schedule  values  used  in 
previous  risky-choice  experiments  with  nonhumans  (e.g.  Bateson  & Kacelnik,  1995;  Davison, 

1969;  Killeen,  1968). 

Although  risky-choice  studies  with  nonhumans  have  suggested  that  the  form  of  the  variable 
delay  distribution  can  be  an  important  determinant  of  risk  sensitivity,  the  effects  of  distributional 
characteristics  on  human  risky  choice  have  not  been  systematically  investigated.  Thus,  in  the 
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present  experiment,  choice  was  assessed  across  bivalued,  rectangular,  normal,  and  exponential 
(positively  skewed)  distributions.  Each  distributions  is  plotted  in  Figure  4.  Included 
on  each  plot  is  the  arithmetic  mean,  harmonic  mean,  and  number  of  delay  intervals  in  the 
distribution.  If  choice  is  consistent  with  previous  research  with  nonhumans,  then  preference  for  the 
VT  schedule  should  increase  as  the  harmonic  mean  decreases.  To  investigate  the  effects  of  the  form 
of  the  variable  distribution  apart  from  the  minimum  delay  value,  all  distributions  had  the  same 
minimum  value  of  1 s. 

One  difference  between  the  risky-choice  experiments  with  delayed  outcomes  conducted 
with  nonhumans  and  the  two  studies  with  humans  (Kohn  et  al.,  1996;  Weiner,  1966)  is  the  nature 
of  the  reinforcer.  In  risky-choice  studies  with  nonhumans,  choices  produced  powerful 
consequences,  normally  food.  In  the  Weiner  experiment  choice  outcomes  were  points  and  in  the 
Kohn  et  al.  (1996)  experiments  choice  outcomes  were  the  appearance  of  symbols  on  the  computer 
screen.  These  reinforcers  were  probably  much  weaker  reinforcers  than  the  food  deliveries  used  in 
the  experiments  with  nonhumans. 

Because  differences  in  the  nature  of  the  outcome  may  contribute  to  the  differences  in  risk 
sensitivity  in  humans  and  nonhumans,  the  present  experiment  used  a presumed  more  potent 
reinforcer,  real  money.  Although  the  Weiner  (1966)  and  Kohn  et  al.  (1996)  experiments  suggest 
that  humans  do  not  show  delay  discounting  for  points  or  symbols,  the  results  of  studies  using 
probabilistic  reinforcers  suggest  that  delay  discounting  may  occur  for  monetary  reinforcers.  Thus, 
by  using  real  money,  risky  choice  in  humans  may  be  more  consistent  with  risky  choice  in 
nonhumans.  All  other  methodological  features  from  Experiment  1,  such  as  extensive  exposure  to 
contingencies,  steady-state  analyses,  etc.,  were  also  in  place  here. 
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Fig.  4.  Probability  distributions  used  in  Experiment  2.  A.M.  is  the  arithmetic 
mean,  H.M.  is  the  harmonic  mean,  and  N is  the  number  of  elements  in  each 
distribution. 


Method 

Subjects 

Four  adult  humans  participated.  Three  subjects  (Subject  203,  a 25  year-old  female. 
Subject  204,  a 26  year-old  male,  and  Subject  206,  a 37  year-old  male)  were  recruited  via  an 
advertisement  in  a local  university  newspaper.  Subject  205  was  female  employee  of  the 
Department  of  Psychology  and  was  approximately  30  years  old.  All  subjects  reported  no  previous 
experience  with  behavioral  research.  At  the  termination  of  the  experiment,  subjects  were  mailed  a 
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check  for  the  money  earned  during  each  session  ($3.00  per  session)  plus  $1 .50  for  each  completed 
session.  Total  earnings  thus  averaged  approximately  $9  per  hour. 

Apparatus 

The  apparatus  was  the  same  as  that  used  in  Experiment  1 , except  that  there  were  no 
counters  and  no  opening  for  a television  monitor  on  the  panel.  Instead,  the  lower  23.5  cm  of  the 
response  panel  was  fitted  with  a solid  piece  of  aluminum. 

Procedure 

Each  session  consisted  of  60  trials.  The  first  30  trials  of  a session  were  forced-choice 
trials  and  the  second  30  were  choice  trials.  During  choice  trials,  both  the  left  and  right  keylights 
were  illuminated  concurrently  and  flashed  according  to  a 0.25  s on-off  cycle.  The  FI  30-s  schedule 
was  correlated  with  a yellow  keylight  and  the  VI  30-s  schedule  was  correlated  with  a green 
keylight.  The  key  position  for  each  schedule  type  was  randomly  determined  each  trial.  A single 
response  stopped  the  keylight  from  flashing,  extinguished  the  alternative  keylight,  and  initiated  the 
schedule  timer.  Completing  the  FI  or  VI  schedule  requirement  extinguished  the  keylight  and 
illuminated  a single,  red  light.  Red  lights  were  illuminated  left  to  right,  beginning  with  the  top  row. 
The  next  trial  began  immediately  following  the  light  illumination.  Forced-choice  trials  were  similar 
to  choice  trials  except  that  only  one  alternative  was  presented.  The  schedule  type  presented  during 
forced-choice  trials  was  determined  randomly  each  trial,  with  the  restriction  that  neither  the  FI  or 
VI  schedule  was  presented  consecutively  on  more  than  three  trials.  Because  the  light  panel 
contained  only  30  red  lights,  the  lights  were  extinguished  following  the  final,  forced-choice  trial  so 
that  they  could  be  re-illuminated  during  the  choice  trials. 

Prior  to  the  first  experimental  session,  the  following  instructions  were  presented  and  read 
to  each  subject: 
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You  have  already  earned  $1.50.  Bonus  earnings  may  be  obtained 
by  turning  on  the  red  lights.  Each  time  a red  light  is  turned  on, 
you  have  earned  an  additional  5 4.  You  can  turn  on  these  lights 
by  pressing  the  response  keys  on  the  front  panel.  Please  remain 
seated.  You  will  be  informed  when  the  session  has  ended. 

Across  conditions,  the  form  of  the  Vl-schedule  distribution  was  systematically  manipulated 
and  was  either  bivalued,  normal,  exponential,  and  rectangular.  Table  2 shows  the  sequence  and 
number  of  sessions  per  condition  for  each  subject.  The  order  of  exposure  to  each  distribution  was 
randomly  determined.  Conditions  were  changed  following  a minimum  of  six  sessions  and  when  the 
number  of  Fl-schedule  choices  was  stable  across  a five-session  block,  as  determined  by  visual 
inspection.  Across  the  first  five  conditions,  choice  in  Subject  205  showed  no  preference  for  either 
alternative  but  an  analysis  of  responding  revealed  a stereotypic  pattern  of  key  pressing  that 
included  responding  on  dark  keys.  To  eliminate  this  pattern,  a 5-s  delay  was  added  between  dark- 
key  responses  and  light  presentations.  This  manipulation  was  successful  in  disrupting  the 


Table  2 


Sequence  and  number  of  sessions  per  condition  for  each  subject  in  Experiment  2. 


Condition 

Subject 

203 

204 

205 

206 

Bivalued 

1(11) 

3(25) 

4(16) 

2(9) 

5(6) 

8(13)* 

5(6) 

Exponential 

2(22) 

2(31) 

1(8) 

3(6) 

8(13) 

5(24) 

8(6) 

7(8)* 

Normal 

4(8) 

4(9) 

2(6) 

1(7) 

7(19) 

9(6)* 

6(6) 

Rectangular 

3(14) 

1(25) 

3(6) 

4(6) 

6(6) 

6(30)* 

7(6) 

* Conditions  conducted  with  a 5-s  delay  programmed  between  responses  on  dark  keys  and  light 
presentations.  See  text  for  details. 
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stereotypic  response  pattern  and  remained  in  effect  during  all  subsequent  conditions.  Only  results 
from  the  conditions  conducted  with  this  delay  are  reported.  These  conditions  are  noted  with  an 
asterisk  in  Table  2. 

Sessions  were  conducted  at  approximately  the  same  hour,  Monday  through  Friday. 
Typically,  one  session  was  conducted  daily.  Several  days  a week,  two  sessions  were  conducted  for 
Subjects  203  and  204.  These  sessions  were  separated  by  approximately  1.5  hr. 

Results 

Figure  5 shows  for  each  subject  the  mean  number  of  choices  for  the  fixed  option  per 
session  across  the  final  five  sessions  of  each  condition.  The  horizontal  line  at  1 5 indicates  the 
indifference  point  between  the  two  options.  Choice  was  defined  as  risk  averse  if  the  number  of 
choices  for  the  fixed  option  fell  above  this  line,  and  risk  prone  if  the  number  of  choices  for  the 
fixed  option  fell  below  it.  Across  all  subjects,  the  form  of  the  variable  distribution  had  no  effect  on 
choice.  For  Subject  203,  choice  was  indifferent  between  the  FI  and  VT  schedules  during  the  first 
condition  (bivalued  distribution)  but  was  slightly  risk  averse  across  the  remaining  conditions  of  the 
experiment.  For  Subject  204,  the  number  of  FI  choices  was  more  variable  than  for  the  other 
subjects  but  showed  no  consistent  preference  for  either  alternative.  For  Subject  205,  choice  was 
also  indifferent  between  the  two  schedules  . For  Subject  206,  choice  showed  a near-exclusive 
preference  for  the  FI  schedule  across  all  conditions. 

The  left  and  right  panels  in  Figure  6 show  mean  response  rates  and  pauses  (time  to  the  first 
response),  respectively,  on  the  FI  and  VI  schedules  during  choice  trials  for  each  subject.  Values 
are  the  mean  of  the  final  five  sessions  of  each  condition.  Because  results  were  similar  across 
exposures  in  most  cases,  results  from  initial  exposures  and  replications  have  been  combined.  The 
vertical  lines  show  the  standard  deviation  of  these  values.  Axes  are  individually-scaled  for  each 
subject.  Across  all  subjects,  the  form  of  the  variable  distribution  had  no  systematic  effect  on 
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Fig.  5.  Mean  number  of  choices  for  the  fixed  option  per  session  across  the  final 
five  sessions  of  each  exposure  to  the  rectangular  (RTG),  exponential  (EXP), 
bivalued  (BV),  and  normal  (NM)  distributions  for  each  subject  in  Experiment  2. 
The  filled  bars  show  values  from  the  initial  exposures  to  each  condition  and  open 
bars  show  values  from  the  replications.  The  horizontal  line  shows  the  indifference 
point  between  the  two  options.  Vertical  lines  show  standard  deviations. 


response  rates  or  pauses.  There  were  large  differences  in  absolute  response  rates  between  subjects; 
response  rates  were  frequently  above  100  responses/min  for  Subject  203,  usually  below  10 
responses/min  for  Subjects  204  and  205,  and  near  2 responses/min  for  Subject  206.  For  Subjects 
203,  204,  and  206,  response  rates  were  also  greater  on  the  VI  than  FI  schedule,  although  for 
Subject  206  the  VI  schedule  was  rarely  chosen.  Responses  rates  for  Subject  205  were  similar  on 
both  the  FI  and  VI  schedules  across  conditions.  Pauses  tended  to  be  slightly  greater  on  the  VI 
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Fig.  6.  Mean  response  rates  (left  panel)  and  pause  durations  (right  panel)  on  the 
fixed-interval  schedule  (filled  bars)  and  variable-interval  schedule  (open  bars) 
across  the  final  five  sessions  of  the  rectangular  (RTG),  exponential  (EXP), 
bivalued  (BV),  and  normal  (NM)  distributions  for  each  subject.  Values  from 
initial  exposures  and  replicated  conditions  have  been  combined.  Axes  have  been 
individually  scaled.  Vertical  lines  show  standard  deviations. 
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schedule  for  Subjects  203,  205,  and  206,  but  were  much  greater  on  the  FI  schedule  for  Subject 
204.  Subject  206  showed  the  longest  overall  pause  durations,  with  pauses  typically  exceeding  30  s. 

The  left  and  right  panels  in  Figure  7 show  the  mean  obtained  delays  to  light  illuminations 
during  forced-choice  and  choice  trials  for  both  the  FI  and  VI  schedules,  averaged  across  the  final 
five  sessions  of  each  condition.  Results  from  initial  exposures  and  replicated  conditions  have  been 
combined.  The  vertical  lines  show  standard  deviations.  For  Subjects  203,  204,  and  205,  obtained 
delays  were  similar  across  distributions  and  across  forced-choice  and  choice  trials.  Obtained 
delays  varied  between  30  and  35  s in  most  cases,  but  were  slightly  above  35  s for  Subject  205.  For 
Subject  206,  during  forced-choice  trials  obtained  delays  varied  between  30  s and  40  s on  the  FI 
schedule  but  were  consistently  longer  (between  40  s and  70  s)  on  the  VI  schedule.  During  choice 
trials,  few  variable  delay  values  were  experienced  for  this  subject  due  to  the  strong  preference  for 
the  fixed  option. 


Discussion 

When  adult  humans  were  presented  with  repeated  choices  between  FI  and  VI  schedules 
having  the  same  mean  value  as  the  form  of  the  VI  distribution  was  manipulated  across  conditions, 
two  subjects  showed  a preference  for  the  FI  schedule  (risk  aversion)  and  two  subjects  showed  no 
strong  preference  for  either  alternative  (risk  indifference).  In  all  subjects,  choice  was  unaffected  by 
the  form  of  the  variable  distribution.  Choice  patterns  were  thus  generally  similar  across  subjects, 
although  there  were  often  large  individual  differences  in  response  rates  and  pauses.  With  the 
exception  of  Subject  206  (see  below)  these  patterns  of  risk  sensitivity  cannot  be  explained  by 
differences  in  average  delays  to  reinforcement  on  the  FI  and  VI  schedule:  Across  forced-choice 
and  choice  trials  obtained  delays  on  both  schedules  were  near  the  programmed  value  (30  s)  in  all 
conditions.  Overall,  these  results  are  consistent  with  the  few  previous  studies  with  humans  (Kohn 
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Fig.  7.  Mean  obtained  delays  to  light  illuminations  on  the  variable  option  per  trial 
across  exposures  to  the  rectangular  (RTG),  exponential  (EXP),  bivalued  (BV), 
and  normal  (NM)  distributions  for  each  subject  during  forced-choice  trials  (left 
panel)  and  choice  trials  (right  panel).  Values  from  the  initial  exposures  and 
replications  of  each  condition  have  been  combined.  The  vertical  lines  show 
standard  deviations.  The  horizontal  line  shows  the  delay  value  of  the  fixed  option 
per  trial  (30  s). 
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et  al.,  1996;  Weiner,  1966)  but  are  inconsistent  with  a large  body  of  nonhuman  research  that  has 
shown  risk  proneness  (e.g.  Hermstein  1964;  Killeen,  1968). 

It  could  be  argued  that  the  invariance  of  choice  across  conditions  indicates  that  behavior 
was  insensitive  to  experimental  contingencies.  For  Subjects  203  and  204,  however,  response  rates 
were  greater  on  the  VI  schedule  across  conditions.  This  finding  indicates  that  responding  was 
sensitive  to  the  different  schedule  contingencies.  For  Subject  203,  choice  was  indifferent  across 
conditions  and  response  rates  were  similar  on  both  the  fixed  and  variable  schedules.  It  is  unclear 
whether  these  results  indicate  an  absence  of  experimental  control,  or  no  strong  preference  for  either 
alternative. 

For  Subject  206,  following  the  first  few  sessions  of  the  experiment,  mean  pause  durations 
increased  beyond  30  s and  remained  above  30  s across  all  conditions  during  both  forced-choice  and 
choice  trials.  Because  the  pause  durations  were  greater  than  30  s,  it  is  unlikely  that  changes  in  the 
form  of  the  variable  distribution  influenced  choice.  Furthermore,  because  only  the  long  delay 
intervals  of  the  VI  schedule  were  experienced,  the  mean  obtained  delays  to  reinforcement  were 
much  greater  on  the  variable  schedule  than  fixed  schedule  in  both  forced-choice  and  choice  trials. 
The  higher  mean  delay  value  experienced  on  the  variable  schedule  could  therefore  explain  the 
exceptionally  strong  risk  aversion  shown  in  this  subject  across  the  experiment.  Thus,  for  Subject 
206,  choice  appeared  to  be  sensitive  to  the  schedule  contingencies  that  were  actually  experienced 
during  experimental  sessions. 

Research  with  nonhumans  has  shown  that  mean  reinforcement  rates  calculated  by 
averaging  methods  that  differentially  weight  short  intervals  of  the  variable  distribution,  (e.g.  the 
harmonic  mean)  provide  a better  description  of  choice  patterns  than  the  arithmetic  mean.  In  the 
present  experiment,  across  all  VI  distributions  the  harmonic  means  of  the  VI  schedule  elements 
were  substantially  lower  than  the  value  of  the  FI  schedule,  yet  in  no  condition  was  the  VI  schedule 
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preferred.  Thus,  the  harmonic  mean,  or  any  averaging  principle  that  predicts  that  the  VI  schedule 
will  have  a lower  reinforcement  rate  or  higher  value  than  the  FI  schedule,  cannot  account  for  the 
present  results.  The  arithmetic  mean  predicts  no  strong  preference  for  either  schedule  and  thus 
provides  a better  description  of  choice  than  the  harmonic  mean,  but  it  only  accounts  for  choice  in 
two  of  the  four  subjects.  The  results  are  also  inconsistent  with  predictions  of  models  such  as  scalar 
expectancy  theory  (SET)  or  the  expectation  of  ratios  (EoR)  that  predict  that  choice  should  be  risk 
prone  across  conditions. 

Choice  patterns  were  also  inconsistent  with  results  of  previous  research  with  nonhumans 
showing  that  the  minimum  delay  value  influences  preference.  In  the  present  experiment,  the 
minimum  delay  value  of  the  VI  schedule  was  held  constant  at  1 s across  all  distributions.  Although 
choice  was  unaffected  by  the  form  of  the  variable  distribution,  the  1-s  value  was  substantially 
shorter  than  the  FI  30-s  schedule  value,  but  in  no  condition  was  the  VI  schedule  preferred. 

It  is  possible  that  choice  was  not  risk  prone  in  the  present  experiment  because  delays  to 
reinforcement  were  too  short  to  produce  significant  delay  discounting.  Across  all  distributions,  the 
delays  to  reinforcement  varied  between  1 s and  105  s.  Although  these  delays  were  similar  to  those 
used  in  experiments  with  nonhumans,  the  delays  used  in  previous  experiments  with  humans  that 
have  shown  delay  discounting  with  hypothetical  monetary  amounts  have  been  substantially  longer 
(e.g.,  Christensen  et  al.,  1998;  Green,  Fry,  & Myerson,  1994;  Green  et  al.,  1999;  Rachlin,  Raineri, 
& Cross,  1991).  For  example.  Green  et  al.  (1999)  showed  delay  discounting  with  hypothetical 
outcomes  across  delays  of  1 month  to  10  years.  Hyten,  Field  and  Madden  (1994),  found  similar 
results  using  points  exchangeable  for  real  money.  In  their  study,  subjects  showed  a preference  for 
a large  point  amount  over  a small  point  amount  when  both  point  amounts  were  exchangeable  for 
money  immediately  following  the  experimental  session.  When  the  delay  to  exchange  the  large 
point  amount  for  money  was  increased  to  three  or  six  weeks,  however,  preference  reversed  in  favor 
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of  the  small  option.  These  findings  suggest  that  if  the  mean  value  of  the  FI  and  VI  schedules  was 
increased,  greater  temporal  discounting  and  greater  risk  proneness  may  occur.  One  problem  with 
using  long  delays,  however,  is  that  when  delays  extend  beyond  the  experimental  session, 
extraneous  variables,  such  as  the  day  of  the  week  on  which  payment  will  occur,  may  interact  with 
choice  contingencies  (Hyten  et  al.  1994). 

It  is  also  possible  that  choice  would  be  more  risk  prone  if  the  choice  outcome  was  a 
consumable  reinforcer,  such  as  food,  rather  than  money.  That  is,  greater  delay  discounting  may 
occur  with  consumable  reinforcers  than  with  nonconsumable  reinforcers.  (This  issue  will  be 
addressed  further  in  the  General  Discussion).  Outside  the  laboratory,  however,  humans  often  do 
show  risk  proneness  when  reinforcers  are  monetary  outcomes.  Developing  a laboratory  procedure 
that  can  reliably  produce  risk  proneness  in  humans  with  monetary  reinforcers  thus  remains  an 
important  goal  of  human  risky-choice  research.  The  purpose  of  Experiment  3,  therefore,  was  to 
investigate  choice  in  humans  with  procedures  that  model  conditions  that  have  produced  both  risk 
aversion  and  risk  proneness  in  nonhumans:  changes  in  an  organism’s  energy  budget. 


EXPERIMENT  3 


ENERGY  BUDGETS 
Overview 

Energy  budget  refers  to  the  relationship  between  an  organism’s  energy  reserves,  short-term 
energy  requirements,  and  the  mean  rate  of  energy  gain  during  a foraging  period.  An  energy  budget 
is  positive  when  the  net  energy  gain  will  exceed  the  requirement,  and  negative  when  the  net  energy 
gain  will  not  exceed  the  requirement.  Behavioral  ecologists  have  shown  that  changes  in  energy 
budget  can  produce  large  shifts  in  risk  sensitivity.  That  is,  choice  may  be  risk  averse  when  energy 
budgets  are  positive,  but  risk  prone  when  energy  budgets  are  negative  (e.g.,  Caraco  et  al.,  1980). 

Although  energy-budget  manipulations  have  been  conducted  with  a variety  of  different 
species,  no  experiments  have  yet  investigated  choice  in  humans  across  changes  in  energy  budget. 
This  is  most  likely  due  to  the  many  ethical  issues  and  practical  problems  involved  with 
manipulating  energy  intake  with  humans,  and  the  lack  of  suitable  alternative  procedures.  Thus,  a 
primary  aim  of  the  present  experiment  was  to  develop  a procedure  for  studying  risky  choice  in 
humans  that  would  model  important  features  of  energy-budget  procedures  without  manipulating 
energy  intake.  This  experiment  would  not  only  help  assess  the  generality  of  energy-budget  effects, 
but  could  also  help  clarify  the  conditions  under  which  choice  of  humans  is  risk  averse  and  risk 
prone. 

It  will  be  useful  first  to  provide  an  overview  of  energy-budget  research.  The  following 
section  contains  a brief  description  of  the  theoretical  assumptions  underlying  energy-budget  models 
and  the  main  experimental  findings. 
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Energy-budget  Research  with  Nonhumans 

The  Energy-Budget  Rule 

The  effects  on  risky  choice  of  an  organism’s  energy  budget  are  predicted  by  optimization 
models  developed  by  behavioral  ecologists.  Optimization  models  are  based  on  the  assumption  that 
fitness  is  maximized  by  optimizing  rates  of  energy  gain.  In  classical  optimization  models,  rate  of 
energy  gain  is  specified  as  an  arithmetic  average,  calculated  by  dividing  the  net  energy  gain  by  the 
total  acquisition  time  (search  and  handling  costs).  This  method  of  calculating  rates  of  energy  gain 
implies  that  the  mean  rate  of  energy  gain,  but  not  variability  in  gain,  influences  choice. 

Several  behavioral  ecologists  have  argued,  however,  that  foraging  choices  should  be 
sensitive  to  variability  (i.e.,  risk  sensitive)  under  some  circumstances  (Caraco,  1980;  Real,  1980). 
One  of  the  first  formal  models  of  risk-sensitive  foraging  was  proposed  by  Stephens  (1981).  Unlike 
most  optimization  models,  this  model  predicts  that  fitness  is  maximized  by  minimizing  the 
probability  of  starvation.  The  model  was  designed  to  predict  choice  in  a forager  faced  with  two 
food  options  who  needs  R units  of  food  to  survive  overnight.  Both  food  options  have  the  same 
mean  value,  but  different  variances.  Stephens  assumed  that  only  one  foraging  choice  occurred 
each  day.  Ignoring  expenditures,  a forager’s  daily  energy  budget  was  described  as: 

f. i*n  + Sn>R  (positive  energy  budget)  (1) 

fj*n  + Sn  < R (negative  energy  budget) 

where  Sn  is  energy  reserves  and  jj  is  mean  food  intake  per  time  interval  and  n is  the  number  of  time 
intervals  in  the  foraging  period. 

To  predict  choice  under  positive  and  negative  energy-budgets,  Stephens  (1981)  assumed 
that  at  the  end  of  a day  a forager  would  have  So  reserves.  This  amount,  determined  by  foraging 
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choices,  was  distributed  normally  with  mean  jj  and  variance  a . Fitness  was  assumed  to  be  a step- 
function  of  reserves.  That  is,  all  reserve  levels  below  So  had  a fitness  value  of  zero  and  all 
reserves  above  So  had  the  same  (non-zero)  value.  Because  So  was  normally  distributed,  R was 
converted  into  a z-score: 


z = (R-fj)/ a 

The  probability  of  survival  at  the  end  of  the  day  was  then  calculated  as: 

P0Sb>/?)  = 1-c t{z) 

where  <£(z)  was  the  cumulative  distribution  of  the  normal  curve.  Because  <£(z)  increases  as  z 
increases,  when  the  energy  budget  is  positive,  probability  of  survival  is  maximized  by  minimizing 
variance  (i.e.,  decreasing  variance  increases  z).  Conversely,  when  the  energy  budget  is  negative, 
probability  of  survival  is  maximized  by  maximizing  variance  (i.e.,  increasing  variance  decreases  z). 

This  model,  called  the  extreme  variance  rule  or  the  z-score  model  (or  sometimes  the 
energy-budget  rule),  can  predict  choice  between  options  having  the  same  mean  value.  It  can  also 
be  extended  to  cases  in  which  options  differ  in  both  their  mean  values  and  variances  (Stephens  & 
Chamov,  1982).  It  should  be  noted  that  the  energy-budget  rule,  which  is  based  upon  overnight 
survival,  is  only  one  type  of  risk-sensitive  foraging  model  (see  McNamara  & Houston,  1992).  The 
energy-budget  rule  will  be  emphasized  here  because  this  model  has  received  the  greatest  amount  of 


empirical  support. 
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Energy-Budget  Experiments 

The  first  experiment  to  demonstrate  shifts  in  risk-sensitivity  with  changes  in  energy  budget 
was  conducted  by  Caraco  et  al.  (1981).  Six  yellow-eyed  juncos  were  given  repeated  choices 
between  two  feeding  stations  delivering  either  a constant  or  variable  number  of  seeds.  The 
probability  distribution  of  the  variable  option  was  bivalued.  Daily  energy  requirements  were 
determined  by  measuring  metabolic  rates  and  the  rate  of  food  intake.  During  positive  energy- 
budget  conditions,  subjects  were  deprived  of  food  for  1 hr  prior  to  experimental  sessions  and  seeds 
were  delivered  at  a mean  rate  that  exceeded  daily  requirements.  During  negative  energy-budget 
conditions,  subjects  were  deprived  for  4 h and  seeds  were  delivered  at  a mean  rate  that  fell  below 
daily  requirements.  Caraco  et  al.  found  that  choice  was  risk  averse  during  positive  energy-budget 
conditions  and  risk  prone  during  negative  energy-budget  conditions.  Thus,  energy  budget 
influenced  risk  sensitivity  in  a manner  consistent  with  the  energy-budget  rule. 

A number  of  subsequent  studies  with  fish  (Croy  & Hughes,  1991;  Young,  Clayton,  & 
Barnard,  1990),  bumblebees  (Cartar,  1991;  Cartar  & Dill,  1990),  shrews  (Barnard  & Brown, 
1985),  and  small  birds,  including  white-crowned  sparrows  (Caraco,  1983)  and  dark-eyed  juncos 
(Caraco,  1981),  have  demonstrated  shifts  in  risk-sensitivity  with  changes  in  energy  budget  (see 
Bateson  & Kacelnik,  1998;  Kacelnik  & Bateson,  1996;  Real  & Caraco,  1986;  for  reviews).  For 
example,  Barnard  and  Brown  (1985)  estimated  daily  energy  budgets  of  wild-caught  shrews  by 
feeding  them  ad  libitum  mealworms  and  measuring  rates  of  food  intake  per  hour.  Shrews  were 
then  deprived  for  1.5  h and  given  repeated  choices  between  a constant  option  delivering  1 food  item 
with  certainty  and  a variable  option  delivering  0 food  items  with  p=.67  or  3 food  items  with  p=.33. 
Energy  budgets  were  manipulated  by  changing  the  duration  of  the  ITI.  Barnard  and  Brown 
showed  that  the  constant  alternative  was  preferred  when  the  energy  budget  was  positive  but  that  the 
variable  alternative  was  preferred  when  the  energy  budget  was  negative. 
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Although  energy  budgets  are  most  often  manipulated  by  changing  deprivation  levels  and 
rates  of  reinforcement,  a unique  study  by  Caraco,  Blanckenhom,  Gregory,  Newman,  Recer,  and 
Zwicker  (1990)  manipulated  energy  requirements  of  yellow-eyed  j uncos  by  changing  the  ambient 
temperature.  Food  consumption  was  measured  at  1 ° , 10°,  and  19°  C.  These  measures  showed 
that  total  consumption  (and  thus  required  energy)  varied  inversely  with  temperature.  During 
experimental  sessions,  the  rate  of  reinforcement  equaled  the  average  rate  of  consumption  during  the 
10°  C condition.  Thus,  during  conditions  in  which  the  temperature  was  1 ° C,  the  energy  budget 
was  negative,  and  during  conditions  in  which  the  temperature  was  19°  C,  the  energy  budget  was 
positive.  Subjects  were  given  repeated  choices  between  options  delivering  a fixed  and  variable 
number  of  seeds.  Choice  was  risk  averse  during  positive  energy-budget  conditions  and  risk  prone 
during  negative  energy-budget  conditions. 

Not  all  experiments  that  have  manipulated  energy  budgets  have  found  complete  shifts  from 
risk  aversion  to  risk  proneness  as  energy  budgets  were  changed  from  positive  to  negative.  In 
several  studies,  only  the  degree  of  risk  aversion  changed  (Hamm  & Shettleworth,  1989;  Ito, 
Takatsuru,  & Saeki,  2000).  That  is,  choice  remained  risk  averse  across  conditions  but  became  less 
risk  averse  under  negative  energy-budget  conditions.  Other  studies  have  shown  no  change  in  risk 
sensitivity  across  changes  in  energy  budget  (Battalio,  Kagel,  & MacDonald,  1985;  Banschbach  & 
Waddington,  1994)  whereas  still  others  have  shown  changes  in  risk  sensitivity  in  the  direction 
opposite  to  that  predicted  by  the  energy-budget  rule  (e.g.,  Hastjarjo,  Silberberg,  & Hursh,  1990; 
Lawes  & Perrin,  1995).  Thus,  while  not  consistent  with  all  available  data,  the  energy-budget  rule 
provides  an  excellent  account  of  risky  choice  in  many  species  and  it  makes  predictions  about  the 
effects  of  energy  budgets  no  other  choice  models  make  (see  Bateson  & Kacelnik,  1998). 
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Energy-budget  Research  with  Humans 

Field  studies 

Because  energy  budgets  can  be  important  determinants  of  risky  choice  in  nonhumans,  it  is 
important  for  both  cross-species  comparisons  and  for  a better  understanding  of  human  behavior  to 
determine  whether  energy  budgets  also  affect  risky  choice  in  humans.  A field  study  by  Kunreuther 
and  Wright  (1979)  has  provided  at  least  qualitative  support  for  the  energy-budget  rule.  Rather  than 
devoting  their  fields  entirely  to  low  variance  food  crops,  poor  farmers  in  Bangledesh  planted  risky 
cash  crops  in  proportions  nearly  equal  to  those  of  rich  farmers.  These  findings  were  interpreted  in 
terms  of  a minimum  subsistence  requirement  (i.e.,  income  level).  Kunreuther  and  Wright  argued 
that  very  wealthy  farmers  who  were  easily  able  to  meet  their  minimum  subsistence  requirement 
planted  risky  (potentially  high-paying)  crops.  Middle  income  farmers  who  had  a moderate  risk  of 
failing  to  meet  their  minimum  requirement  preferred  low  variance  food  crops.  The  low-income 
farmers,  however,  who  had  the  largest  probability  of  not  meeting  their  minimum  requirement 
planted  riskier  crops  to  avoid  poverty.  If  the  crop  returns  did  not  meet  their  minimum  requirement, 
they  were  forced  to  borrow  money  from  others  or  face  starvation. 

Although  these  findings  are  broadly  consistent  with  the  energy-budget  rule,  field  studies 
lack  the  quantitative  rigor  needed  for  strong  tests  of  the  model.  That  is,  in  the  field  it  is  difficult  to 
manipulate  and  assess  variables  that  determine  energy  budget  and  it  is  often  not  practically  feasible 
to  eliminate  extraneous  variables.  In  the  laboratory,  however,  variables  can  be  carefully 
controlled,  permitting  more  precise  tests  of  optimality  models  such  as  the  energy-budget  rule.  As 
described  above,  no  laboratory  studies  have  yet  manipulated  energy  budgets  in  humans,  possibly 
owing  to  the  ethical  difficulties  of  establishing  negative  energy  budgets.  It  may  be  possible, 
however,  to  study  choice  in  humans  under  non  life-threatening  conditions  with  a experimental 
procedure  that  models  energy-budget  manipulations. 
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Hypothetical  life-death  choices 

Several  researchers  have  attempted  to  model  risky  choice  in  humans  in  life-threatening 
situations  by  presenting  subjects  with  choices  between  hypothetical  life-death  options  (e.g.,  Wang, 
1995,  1996a,  1996b;  Wang  & Johnston,  1995).  Wang  and  Johnston  (1995)  and  Wang  (1996a), 
for  example,  presented  subjects  with  hypothetical  choices  between  saving  the  lives  of  either  a 
certain  or  variable  number  of  people  as  the  size  of  the  hypothetical  group  was  manipulated.  Both 
studies  showed  that  choice  was  more  risk  seeking  when  the  group  size  was  small  than  when  the 
group  size  was  large.  Wang  interpreted  these  results  in  terms  of  the  subjects’  own  survival, 
arguing  that  throughout  human  history,  an  individual’s  own  chance  of  survival  was  directly  related 
to  the  survival  of  a minimum  number  of  people  in  the  individual’s  social  group.  Thus,  if  an 
individual  was  faced  with  a risky  choice  involving  the  life  and  death  of  members  of  their  social 
group,  choice  should  be  risk  prone  if  choice  of  the  certain  option  would  fail  to  save  a sufficient 
number  of  people.  Wang  (1996b)  compared  the  results  of  these  studies  to  those  found  in  risk- 
sensitive  foraging  experiments,  noting  that  whenever  an  organism  is  faced  with  a risky  choice 
involving  a life-death  decision,  choice  should  be  risk  seeking  if  choice  of  the  certain  outcome  could 
lead  to  the  individual’s  own  death. 

Although  the  results  of  these  studies  show  that  risky  choice  in  humans  may  be  influenced 
by  variables  related  to  survival,  the  procedures  are  limited  in  their  ability  to  model  energy-budget 
procedures.  First,  the  life-death  outcomes  are  necessarily  hypothetical.  It  is  unclear  whether 
hypothetical  outcomes  produce  the  same  patterns  of  risk  sensitivity  as  real  outcomes,  particularly 
when  outcomes  are  related  to  survival.  Second,  because  minimum  requirements  are  not  assessed  or 
manipulated  prior  to  the  presentation  of  choice  options,  it  is  difficult  to  predict  whether  risk 
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proneness  or  risk  aversion  should  occur.  Because  of  these  limitations,  procedures  using 
hypothetical  life-death  choices  may  be  unsuitable  for  studying  energy-budget  effects  in  humans. 

Monetary  outcomes 

One  alternative  to  using  hypothetical  life-death  choices  to  study  the  effects  of  energy 
budget  with  humans  is  to  use  real  outcomes,  such  as  money,  that  are  not  important  for  immediate 
survival.  That  is,  it  may  be  possible  to  conduct  the  same  types  of  experimental  manipulations  with 
humans  as  conducted  in  energy-budget  experiments  with  nonhumans  using  money  rather  than 
energy  as  a consequence. 

For  example,  one  might  manipulate  energy  budgets  by  changing  the  rate  of  reinforcement. 
Rachlin  et  al.  (1986)  presented  subjects  with  ten  choices  between  two  options  delivering 
hypothetical  amounts  of  money  with  a high  and  low  probability.  Trials  for  half  of  the  subjects 
were  presented  with  no  ITI  whereas  trials  for  the  other  half  of  the  subjects  were  separated  by  a 1.5 
min  ITI.  Because  decreasing  the  rate  of  trial  presentations  produces  concomitant  changes  in  rates 
of  reinforcement,  choice  should  be  risk  averse  when  trials  are  presented  at  a high  rate  (i.e.,  positive 
energy-budget  conditions)  and  risk  prone  when  trials  are  presented  at  a low  rate  (i.e.,  negative 
energy-budget  conditions).  Contrary  to  these  predictions,  choice  of  subjects  in  the  long  ITI  group 
was  more  risk-averse  than  choice  of  subjects  in  the  short  ITI  group.  Experiments  by  Rachlin  and 
Siegel  (1994)  and  Silberberg  et  al.  (1988)  showed  similar  effects. 

Another  way  to  model  changes  in  energy-budget  would  be  to  manipulate  monetary 
holdings  (i.e.,  reserves).  Using  a procedure  similar  to  that  of  Rachlin  et  al.  (1986),  Silberberg  et 
al.  (1988)  presented  40  undergraduates  with  choices  between  two  options  delivering  a hypothetical 
amount  of  money  with  either  a high  or  low  probability.  Half  of  the  subjects  were  told  that  they 
would  start  the  experiment  with  a hypothetical  $10,  and  half  were  told  that  they  would  start  with  a 
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hypothetical  $10,000.  Because  presenting  large  amounts  of  money  increases  reserves,  choice  of 
subjects  in  the  $10,000  group  should  be  more  risk  averse  than  choice  of  subjects  in  the  $10  group. 
Across  trials,  choice  patterns  were  inconsistent  with  these  predictions:  Choice  of  subjects  in  the 
$10,000  was  less  risk  averse  than  choice  of  subjects  in  the  $10  group. 

The  risk  aversion  shown  in  these  experiments  is  inconsistent  with  what  would  be  predicted 
by  analogous  energy-budget  manipulations.  However,  these  experiments  were  not  designed  to 
model  energy-budget  effects  and  it  unclear  whether  conditions  analogous  to  positive  and  negative 
energy  budgets  were  actually  produced.  Energy  budgets  are  defined  in  terms  of  the  relationship 
between  energy  reserves,  rates  of  energy  intake,  and  energy  requirements.  In  the  Rachlin  et  al.  and 
Silberberg  et  al.  experiments,  although  rate  of  reinforcement  (analogous  to  rates  of  food  delivery) 
and  monetary  holdings  (analogous  to  energy  reserves)  were  manipulated,  variables  analogous  to 
energy  requirements  were  not  assessed.  Without  assessing  requirements,  it  is  impossible  to 
determine  whether  changes  in  reinforcement  rate,  or  the  receipt  of  $10  or  $10,000  (even  if  actually 
delivered  to  subjects)  would  create  conditions  analogous  to  positive  or  negative  energy  budgets. 
Thus,  the  present  experiment  attempted  to  model  energy-budget  procedures  by  substituting 
monetary  earnings,  monetary  reserves,  and  a monetary  earnings  requirement  for  energy  gains, 
energy  reserves,  and  a daily  energy  requirement. 

Dynamic  Optimization  Models 

Because  the  energy-budget  rule  predicts  only  one  pattern  of  risk  sensitivity  across  an  entire 
foraging  period  (i.e.,  exclusive  preference  for  the  fixed  or  variable  options),  it  has  been  called  a 
static  optimization  model  (see  Kacelnik  & Bateson,  1996;  Krebs  & Kacelnik,  1991).  It  has  been 
argued,  however,  that  switching  between  the  fixed  and  variable  options  as  a function  of  current 
energy  gains  may  sometimes  be  a better  strategy  than  persisting  with  a single  choice  option  (e.g. 
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Houston,  1991;  Houston  & McNamara,  1982;  McNamara  & Houston,  1987;  Real  and  Caraco, 
1986).  For  example,  if  a forager  whose  energy-budget  is  currently  negative  experiences  a period  of 
high  gain,  then  switching  from  the  variable  option  to  the  fixed  option  may  increase  the  probability 
of  survival.  Thus,  another  goal  of  the  present  research  was  to  provide  a detailed  analysis  of  choice 
patterns  in  humans  within,  as  well  as  across,  choice  periods. 

Current  energy  budget  can  be  regarded  as  a state  variable.  When  choice  varies  as  a 
function  of  current  state,  and  state  varies  as  a function  of  previous  choices,  then  a dynamic 
optimization  model,  rather  than  a static  model,  is  required  to  predict  optimal  choice  patterns 
(Houston  & McNamara,  1988;  Mangel  & Clark,  1988).  Because  dynamic  optimization  models 
are  designed  to  predict  local  regularities  in  choice  patterns,  they  provide  a more  detailed  description 
of  behavior  than  static  models  designed  to  predict  more  global  outcomes  (Krebs  & Kacelnik, 

1991).  The  major  features  of  dynamic  optimization  modeling  are  outlined  briefly  below. 

As  with  static  optimization  models,  an  important  feature  of  a dynamic  optimization  model 
is  the  function  relating  fitness  to  the  organism’s  state  at  the  end  of  a time  period,  called  the  terminal 
fitness  function.  Once  the  terminal  fitness  function  is  specified,  the  total  time  period  over  which 
choice  is  assessed,  T,  is  divided  into  n discrete  time  intervals,  denoted  by  /,  during  which  a choice 
occurs.  Because  the  optimal  choice  at  each  value  of  t and  each  state  (i.e.,  level  of  energy  reserves), 
x,  depends  on  the  outcome  of  future  choices,  optimal  choices  are  computed  backwards  from  T-l  to 
T-n.  That  is,  at  each  state,  optimal  choice  is  first  computed  during  the  final  time  interval,  T-l,  by 
determining  which  choice  yields  the  highest  fitness  value. 

Optimal  choice  is  determined  by  the  state  dynamics  proposed  in  a particular  model.  The 
state  dynamics  are  the  calculations  that  determine  the  expected  change  in  state  for  each  choice 
option  at  the  current  state  value.  The  state  dynamics  include  variables  such  as  the  mean  gain, 
probability  of  gain,  and  cost.  The  fitness  value  associated  with  each  state  at  T-l  is  provided  by  the 
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terminal  fitness  function.  The  choice  option  yielding  the  highest  fitness  is  designated  as  the  optimal 
choice.  At  T-2,  the  optimal  choice  at  each  state  is  computed  by  determining  which  choice  option 
yields  the  highest  expected  fitness  at  T-l,  assuming  that  at  T-l  the  optimal  choice  occurred.  These 
calculations  are  continued  for  each  preceding  time  interval,  producing  an  optimal  choice  matrix, 
sometimes  called  the  optimal  policy,  which  lists  the  optimal  choice  for  each  (/,  x)  combination 
(Houston  & McNamara,  1988). 

McNamara  and  Houston  (1987,  1992)  have  described  an  optimal  policy  of  a forager 
needing  sufficient  reserves  to  survive  overnight.  In  their  example,  the  two  choice  options  had 
identical  means,  but  different  variances.  The  terminal  fitness  function  was  assumed  to  be  a step 
function,  with  fitness  being  0 if  reserves  were  below  requirements  and  1 if  reserves  were  above 
requirements.  They  reported  that  the  optimal  policy  could  be  described  in  terms  of  a switching  line 
specified  by: 


x + iiT-t)  = R (2) 

where  x is  the  current  state,  y is  the  mean  rate  of  energy  gain  per  time  interval,  T is  the  total 
number  of  time  intervals,  t is  the  current  time  interval,  and  R is  the  energy  requirement.  In  a plot 
of  reserves  versus  time,  risk  aversion  is  optimal  above  this  line  and  risk  proneness  is  optimal  below 
this  line. 

This  model  (Equation  2)  is  similar  to  the  energy-budget  rule  (Equation  1 ).  The  primary 
difference  between  these  models  is  that  the  energy-budget  rule  predicts  only  a single  choice  at  the 
start  of  the  foraging  period,  but  the  dynamic  model  predicts  choice  across  the  foraging  period. 

Whether  static  or  dynamic  models  are  used  to  predict  choice  depends  on  the  choice  context 
(Bateson  & Kacelnik,  1 998).  The  static  model  is  better  suited  to  predict  single  choices,  such  as  the 
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choice  of  a food  patch,  whereas  the  dynamic  models  is  better  suited  to  predict  sequences  of  choices, 
such  as  choice  of  food  items  within  a patch. 

Dynamic  optimization  models  may  also  be  used  to  calculate  the  fitness  costs  (i.e.,  loss  in 
fitness)  from  choosing  the  non-optimal  alternative  (Houston  & McNamara,  1988;  McNamara  & 
Houston,  1 986).  This  cost,  called  the  canonical  cost,  is  calculated  by  subtracting  the  expected 
terminal-fitness  value  of  the  non-optimal  choice  from  the  expected  terminal-fitness  value  of  the 
optimal  choice.  If  the  optimal  choice  is  selected,  the  canonical  cost  is  zero.  This  comparison  is 
useful  because  behaving  sub-optimally  may  be  more  costly  at  some  time  and  state  values  than 
others.  McNamara  and  Houston  argued  that  the  canonical  cost  provides  a common  scale  by  which 
different  choices,  even  those  with  different  types  of  consequences  (e.g.  food  or  predator  avoidance), 
can  be  evaluated. 

Kacelnik  and  Bateson  (1996)  have  noted  that  it  is  difficult  to  make  quantitative  predictions 
about  optimal  behavior  from  risk-sensitive  foraging  models.  In  most  experiments,  there  are  a 
number  of  uncontrolled  variables.  For  example,  it  is  often  difficult  to  determine  precisely  energy 
gains  and  energy  expenditures.  Furthermore,  it  is  difficult  to  determine  the  extent  to  which  shifts 
from  risk  aversion  to  risk  proneness  influence  fitness. 

One  advantage  of  laboratory  procedures  is  that  because  the  current  state  (accumulated 
earnings),  the  energy  gain  (reinforcer  magnitude),  the  energy  requirement  (earnings  requirement), 
the  energy  budget,  and  the  terminal  fitness  function  (total  earnings),  can  be  precisely  specified  and 
measured,  exact  predictions  can  be  made  about  optimal  responding.  Thus,  in  the  present 
experiment,  a dynamic  optimization  model  was  developed  to  predict  sequences  of  choices  within  a 
choice  period.  Choices  were  evaluated  both  in  relation  to  the  predictions  of  the  dynamic  model, 
and  in  relation  to  the  canonical  cost  of  choosing  the  non-optimal  alternative. 
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Specific  Rationale  for  Experiment  3 

A primary  goal  of  Experiment  3 was  to  develop  a procedure  with  monetary  outcomes  to 
investigate  risky  choice  in  humans  across  manipulations  analogous  to  those  used  to  in  energy- 
budget  experiments  with  nonhumans.  The  results  of  this  experiment  should  be  relevant  not  only  to 
cross-species  generalities  of  risky  choice,  but  also  to  energy-budget  models  designed  to  predict 
optimal  choice  in  foragers. 

Humans  were  given  choices  between  options  that  delivered  a fixed  and  variable  number  of 
points  exchangeable  for  money.  Choices  were  presented  in  blocks  of  five  trials  designed  to 
simulate  a daily  foraging  period.  To  model  a daily  energy  requirement,  a monetary  earnings 
requirement  was  arranged  for  each  choice  period.  At  the  end  of  a choice  period,  if  the  earnings 
requirement  was  met,  the  subject  was  allowed  to  keep  the  accumulated  earnings.  If  the  earnings 
requirement  was  not  met,  the  earnings  were  lost.  Meeting,  or  failing  to  meet,  the  earnings 
requirement  thus  produced  consequences  that  modeled  the  binary  life-death  outcomes  of  meeting  or 
failing  to  meet  a daily  energy  requirement.  Similar  to  energy-budget  experiments  conducted  with 
nonhumans,  during  positive  energy-budget  conditions  exclusive  preference  of  the  fixed  option 
would  meet  the  earnings  requirement,  but  during  negative-energy-budget  conditions  exclusive 
preference  of  the  fixed  option  would  not  meet  the  earnings  requirement.  With  this  arrangement,  it 
was  possible  to  examine  the  predictions  of  the  energy-budget  rule  with  humans.  If  choice  in 
humans  is  consistent  with  the  model’s  predictions,  choice  would  be  risk  averse  during  positive 
energy-budget  conditions  and  risk  prone  during  negative  energy-budget  conditions. 

Another  purpose  of  the  present  experiment  was  to  analyze  within-block  choice  patterns  by 
comparing  choices  to  the  predictions  of  a dynamic  optimization  model.  Thus,  a dynamic 
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optimization  model  was  constructed  to  predict  optimal  choices  across  trials  of  a block  as  a function 
of  accumulated  points. 

As  in  Experiments  1 and  2,  choice  outcomes  were  real.  In  addition,  because  Experiment  1 
showed  that  choice  between  fixed  and  variable  reinforcer  amounts  was  not  influenced  by  the  form 
of  the  variable  distribution,  only  the  bivalued  distribution  was  used. 


Method 

Subjects 

The  participants  were  three  adult  humans  recruited  via  an  advertisement  in  a local 
university  newspaper.  Subject  33 1 was  a 20  year-old  male  and  Subjects  33 1 and  332  were  both  22 
year-old  females.  None  had  any  previous  experience  with  behavioral  research.  Point  earnings 
were  exchanged  for  cash  immediately  following  each  session.  Subjects  were  also  given  a receipt 
after  each  session  in  the  amount  of  $1 .50.  At  the  termination  of  the  experiment,  a check  for  the 
sum  of  these  receipts  was  mailed  to  the  subject.  Overall  earnings  averaged  near  $6.00  per  hour. 
Apparatus 

The  apparatus  was  the  same  as  in  Experiment  1 (with  the  modifications  described  for 
Subject  218)  except  that  the  4-digit  electromechanical  counter  was  replaced  by  a 6-digit  electrical 
counter  measuring  3 cm  by  6 cm,  and  a second,  identical  counter  was  mounted  1.8  cm  below  it. 
Procedure 

A session  consisted  of  12  blocks  of  five  trials.  The  first  six  blocks  of  a session  were 
forced-choice  trials  and  the  second  six  blocks  were  choice  trials.  At  the  start  of  each  block,  the  top 
(trial)  counter  was  set  to  zero.  The  start  of  each  trial  was  signaled  by  the  illumination  of  the  center 
key.  During  choice  blocks,  a single  response  on  the  red  center  key  extinguished  the  keylight  and 
illuminated  both  side  keys  which  flashed  according  to  a 0.25  s on-off  cycle.  The  fixed  and  variable 
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alternatives  were  correlated  with  yellow  and  green  key  lights,  respectively.  Five  responses  on  one 
of  the  keys  extinguished  both  keylights  and  produced  points.  If  the  fixed  option  had  been  selected, 
two  points  were  added  to  the  trial  counter  and  if  the  variable  option  had  been  selected,  one  or  three 
points  (p=.5)  were  added  to  the  trial  counter.  The  key  position  of  the  fixed  and  variable  option  was 
randomly  determined  each  trial.  Following  point  delivery  on  the  fifth  trial  of  a block,  if  the  number 
of  points  on  the  trial  counter  equaled  or  exceeded  the  point  requirement,  the  points  on  the  trial 
counter  were  added  to  the  lower  (block)  counter.  If  the  number  of  points  on  the  trial  counter  was 
less  than  the  point  requirement,  no  points  were  added  to  the  block  counter.  The  trial  counter  was 
then  reset  and  the  next  block  began  immediately. 

Forced-choice  blocks  were  similar  to  choice  blocks  except  that  only  one  keylight  was 
illuminated  and  across  all  five  trials  only  the  fixed  or  variable  option  was  presented.  The  schedule 
type  was  randomly  determined  each  block  with  the  restriction  that  three  blocks  presented  only  the 
fixed  option  and  three  blocks  presented  only  the  variable  option.  Across  forced-choice  and  choice 
trials,  pressing  a dark  key  or  switching  between  keys  reset  the  FR  5 schedule,  such  that  five 
consecutive  responses  were  required  to  produce  points. 

The  point  requirement  was  manipulated  across  conditions.  During  Positive  Energy-Budget 
conditions,  the  point  requirement  was  10  points  (R=10).  Under  these  conditions,  exclusive 
preference  for  the  fixed  option  would  meet  the  requirement;  exclusive  preference  for  the  variable 
option  would  meet  the  requirement  half  the  time,  on  average.  During  Negative-Energy  Budget 
conditions,  the  point  requirement  was  either  12  points  (R=12)  or  13  points  (R=13).  Exclusive 
preference  for  the  fixed  option  would  not  meet  the  requirement  under  either  of  these  conditions; 
exclusive  preference  for  the  variable  option  would  occasionally  meet  the  criterion  (.19  probability). 

The  following  instructions  were  posted  to  the  right  of  the  response  panel  and  were  read  to 
the  subject  prior  to  the  first  experimental  session; 
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You  may  earn  points  by  pressing  the  response  keys  when  lit. 

Press  only  one  key  at  a time.  Each  point  displayed  on  the  lower, 
right  counter  is  worth  2.5  4.  Please  remain  seated.  You  will  be 
informed  when  the  session  is  over. 

Table  3 shows  the  sequence  and  number  of  sessions  per  condition  for  each  subject.  All 
subjects  were  first  exposed  to  Positive  Energy-Budget  (R=10)  conditions.  Conditions  were 
changed  after  a minimum  of  five  sessions  and  when  the  number  of  choices  for  the  fixed  option  was 
stable  across  a three-session  block,  as  determined  by  visual  inspection.  Sessions  were  conducted  at 
approximately  the  same  hour,  Monday  through  Friday.  Typically,  two  sessions  were  conducted 
daily  with  sessions  separated  by  a brief  (approximately  2 min)  break. 

Because  Subject  333  showed  little  sensitivity  to  energy-budget  contingencies  during  the 
first  exposure  to  Positive  (R=  1 0)  and  Negative  Energy-Budget  (R=  13)  conditions,  during  the 
second  replication  of  the  Positive  Energy-Budget  (R=10)  condition  the  number  of  forced-choice 
blocks  per  session  was  increased  from  six  to  ten.  Across  sessions,  the  number  of  forced-choice 
blocks  was  gradually  reduced,  and  across  the  final  five  sessions  of  the  condition  the  number  of 
forced-choice  blocks  was  six. 


Table  3 


Sequence  and  number  of  sessions  per  condition  for  each  subject  in  Experiment  3. 


Energy-budget 

Condition 

Subject 

331 

332 

333 

Positive  (R=10) 

1(10) 

1(7) 

1(9) 

3(19) 

3(12) 

3(16) 

5(6) 

Negative  (R=12) 

2(8) 

2(6) 

Negative  (R=  13) 

4(5) 

4(5) 

2(10) 

4(7) 
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Results 

Across-block  choices 

Figure  8 shows  the  mean  number  of  choices  for  the  fixed  option  across  the  final  three 
sessions  of  positive  and  negative  energy-budget  conditions.  The  horizontal  line  indicates  the 
indifference  point  between  the  two  options  and  the  vertical  lines  show  standard  deviations.  For  all 
subjects,  the  across-block  choices  showed  sensitivity  to  energy-budget  conditions.  Subjects  331 
and  332  showed  a similar  pattern  of  responding.  During  the  first  exposure  to  the  Positive  Energy- 
Budget  (R=10)  condition,  choice  was  strongly  risk  averse.  The  mean  number  of  choices  for  the 
fixed  option  during  the  final  three  sessions  of  the  condition  was  approximately  29.3  and  29.0  for 
Subjects  33 1 and  332,  respectively.  Choice  was  also  risk  averse  during  the  second  exposure  to 
Positive  Energy-Budget  (R=10)  conditions,  but  there  was  greater  session-to-session  variability  in 
choices.  Across  the  final  three  sessions,  the  mean  number  of  choices  for  the  fixed  option  for  these 
two  subjects  was  28.0  and  22.7,  respectively.  Choice  was  risk  prone  during  both  exposures  to 
negative  energy  budget  conditions.  During  the  Negative  Energy-Budget  (R=  1 2)  condition,  the 
number  of  choices  for  the  fixed  option  for  Subjects  33 1 and  332  was  13.3  and  12.3,  and  during  the 
Negative  Energy-Budget  (R=13)  condition,  in  which  the  point  requirement  was  most  stringent,  the 
mean  number  of  choices  for  the  fixed  option  was  6.7  and  4.7,  respectively. 

Choice  of  Subject  333  was  also  sensitive  to  positive  and  negative  energy-budget 
conditions,  but  the  sensitivity  was  somewhat  weaker  than  in  the  other  two  subjects.  During  the 
first  exposure  to  both  the  Positive  Energy-Budget  (R-l  0)  condition  and  the  Negative  Energy- 
Budget  (R-13)  condition,  the  number  of  choices  for  the  fixed  option  was  similar.  Across  the  final 
three  sessions  of  these  conditions,  the  mean  number  of  choices  for  the  fixed  option  was  17.7  and 
1 7.0,  respectively.  Thus,  during  the  second  replication  of  the  Positive  Energy-Budget  (R=10) 
condition,  the  number  of  forced-choice  trials  was  increased  to  50  for  several  sessions,  and  was  then 
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Fig.  8.  Mean  number  of  choices  for  the  fixed  option  per  session  across  the  final 
three  sessions  of  positive  (POS)  and  negative  (NEG)  energy-budget  conditions  for 
each  subject  in  Experiment  3.  Vertical  lines  show  standard  deviations.  The 
horizontal  line  indicates  the  indifference  point  between  the  two  options. 
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gradually  decreased  across  sessions  to  its  original  value  of  30.  This  manipulation  produced  greater 
sensitivity  to  energy-budget  contingencies.  Across  the  final  three  sessions  of  the  subsequent  two 
exposures  to  Positive  Energy-Budget  (R=10)  conditions,  the  mean  number  of  choices  for  the  fixed 
option  was  28.3  and  26.5,  comparable  to  the  other  two  subjects.  The  number  of  choices  for  the 
fixed  option  decreased  during  the  Negative  Energy-Budget  condition  (R=13),  although  choice 
remained  slightly  risk  averse.  During  the  final  three  sessions,  the  mean  number  of  choices  for  the 
fixed  option  was  16.7. 

Figure  9 shows  the  mean  number  of  points  earned  during  choice  trials  across  the  final  three 
sessions  of  each  condition.  Point  earnings  varied  substantially  across  positive  and  negative  energy- 
budget  conditions.  For  Subjects  33 1 and  332,  point  earnings  were  high  during  both  exposures  to 
Positive  Energy  Budget  (R=10)  conditions.  During  the  first  exposure,  the  mean  number  of  earned 
points  was  60.0  for  both  subjects,  but  was  slightly  lower  during  the  second  exposure  due  to  the 
greater  number  of  variable  choices.  Point  earnings  were  much  lower  during  both  negative  energy- 
budget  conditions.  The  mean  number  of  earned  points  was  28.0  and  45.0  across  the  final  three 
sessions  of  the  Negative  Energy-Budget  (R-13)  condition,  and  17.3  and  8.7  across  the  final  three 
sessions  of  the  Negative  Energy-Budget  (R- 1 2)  condition  for  Subjects  33 1 and  332,  respectively. 

For  Subject  333,  because  choice  showed  little  sensitivity  to  energy-budget  contingencies 
during  the  first  exposures  to  both  the  Positive  Energy  Budget  (R=10)  and  Negative  Energy-Budget 
(R=13)  conditions,  point  earnings  were  lower  than  for  the  other  two  subjects.  During  the 
subsequent  two  exposures  to  Positive  Energy-Budget  (R=  1 0)  conditions,  however,  mean  point 
earnings  were  also  high,  averaging  57.0  and  54.0  points  across  the  final  three  sessions.  During  the 
final  exposure  to  the  Negative  Energy-Budget  (R=13)  condition,  point  earnings  were  very  low,  and 
across  the  final  three  sessions,  no  points  were  earned. 
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Fig.  9.  Mean  number  of  points  earned  during  choice  trials  across  the  final  three 
sessions  of  positive  (POS)  and  negative  (NEG)  energy-budget  conditions  for  each 
subject.  Vertical  lines  show  standard  deviations. 


Within-block  choices 


The  number  of  fixed  and  variable  choices  and  the  number  of  accumulated  points  across 
successive  trials  of  a block  is  shown  in  Appendix  A for  each  subject  over  the  final  three  sessions  of 
a condition.  The  within-block  choice  patterns  were  analyzed  in  accordance  with  the  predictions  of 
a dynamic  optimization  model.  The  model  generated  expected  earnings  for  fixed  and  variable 
choices  at  each  number  of  accumulated  points  for  each  trial  of  a block.  The  choice  sequence 
yielding  the  highest  expected  earnings  was  designated  as  the  optimal  choice.  (A  description  of  how 
the  model  generated  expected  earnings,  and  thus  optimal  choices,  is  presented  in  Appendix  B). 

Tables  4,  5,  and  6 show  the  expected  earnings  associated  with  selecting  the  fixed  and 
variable  option  at  each  number  of  accumulated  points  during  each  trial  of  a block  for  the  Positive- 
Energy  Budget  (R=10),  Negative-Energy  Budget  (R=12),  and  Negative-Energy  Budget  (R=13) 
conditions.  The  optimal  choice  at  each  trial  within  a block  is  underlined.  The  final  column  in  each 
table  shows  the  values  of  the  terminal  fitness  function,  designated  R(x).  The  terminal  fitness 
function  specified  that  the  number  of  points  earned  at  the  end  of  a block  would  equal  zero  if  the 
number  of  accumulated  points  was  below  the  requirement,  and  would  equal  the  point  earnings  if 
the  number  of  accumulated  points  was  equal  or  greater  than  the  requirement.  The  maximum  value 
of  R(\)  was  1 5~the  maximum  number  of  points  that  could  be  earned  per  block.  Overall,  the  tables 
show  that,  consistent  with  the  model  described  by  McNamara  and  Houston  (1987),  preference  for 
the  risky  option  was  predicted  only  when  the  expected  earnings  were  below  the  requirement. 
Preference  for  the  fixed  option  was  predicted  when  the  expected  earnings  equaled  the  requirement. 
When  the  expected  earnings  of  the  fixed  and  variable  option  were  identical,  no  particular  choice 
was  designated  as  optimal. 

Also  included  in  Tables  4,  5,  and  6 are  the  canonical  costs,  or  the  losses  in  expected  point 
earnings  from  choosing  the  non-optimal  choice.  These  values  were  calculated  by  subtracting  the 


Expected  point  earnings  for  selections  of  the  fixed  (F)  and  variable  (V)  option  at  each  number  of  accumulated  points  for  the  Positive  Energy- 
Budget  (R=10)  condition.  The  optimal  choice  is  underlined.  R(x)  indicates  the  terminal  fitness  function  (the  number  of  points  delivered  for  each 
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Expected  point  earnings,  canonical  costs,  and  terminal  fitness  function,  R(x), for  the  Negative  Energy-Budget  (R=12)  condition.  All  other  details  as 
in  Table  4. 
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Expected  point  earnings,  canonical  costs,  and  terminal  fitness  function,  R(x),  for  the  Negative  energy-budget  (R=13)  condition.  All  other  details  as 
in  Table  4. 
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expected  earnings  of  non-optimal  choice  from  the  expected  earnings  of  the  optimal  choice.  If 
choice  was  sensitive  to  the  differential  outcomes  of  choosing  optimal  and  non-optimal  choices,  then 
deviations  from  the  optimal  pattern  should  vary  inversely  with  the  cost  of  such  deviations. 

Tables  7,  8,  and  9 show  for  each  subject  the  proportion  of  total  choices  consistent  with  the 
model’s  predictions  at  each  trial  and  point  combination  for  each  condition.  Only  choices  from  the 
final  three  sessions  of  a condition  are  included.  Overall,  choices  were  generally  consistent  with 
model’s  predictions.  For  the  two  subjects  showing  the  greatest  sensitivity  to  energy-budget 
conditions  (Subjects  33 1 and  332),  the  within-block  choices  were  similar.  The  proportion  of 
choices  consistent  with  predictions  were  frequently  high,  but  choices  were  more  consistent  with 
predictions  during  the  Positive  Energy-Budget  (R=10)  than  during  the  Negative  Energy-Budget 
(R=12)  and  Negative  Energy-Budget  (R=12)  conditions.  Across  all  conditions,  choices  became 
more  consistent  with  the  model’s  predictions  across  trials  within  a block.  In  other  words,  choices 
were  more  likely  to  deviate  from  the  optimal  pattern  earlier  than  later  in  a block.  The  patterns  of 
deviations  were  also  similar  for  these  two  subjects.  Most  deviations  in  positive  energy-budget 
conditions  were  the  result  of  variable  choices  early  in  the  block  when  preference  for  the  fixed 
option  was  predicted,  and  most  deviations  in  negative-energy-budget  conditions  were  the  result  of 
choices  for  the  fixed  option  early  in  the  block  when  preference  for  the  variable  option  was 
predicted. 

For  Subject  333,  within-block  choices  are  presented  only  for  two  Positive  Energy-Budget 
(R=T0)  conditions  and  the  one  Negative  Energy-Budget  (R=13)  condition  that  followed  the 
additional  exposure  to  the  forced-choice  trials.  Within-block  choices  were  generally  consistent 
with  predictions  during  the  Positive  Energy-Budget  (R=10)  conditions,  but  were  often  inconsistent 
with  predictions  during  the  Negative  Energy-Budget  (R=13)  conditions.  Choices  showed  only  a 
slight  tendency  to  become  more  consistent  with  predictions  across  a block.  The  pattern  of 
deviations  is  also  more  difficult  to  characterize  for  this  subject.  In  both  positive  and  negative 


Proportion  of  fixed  (F)  and  variable  (V)  choices  consistent  with  the  model’s  predictions  across  the  final  3 sessions  of  a condition  for  each  trial  of  a 
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Table  7 cont. 
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Proportion  of  fixed  (F)  and  variable  (V)  choices  consistent  with  the  model’s  predictions  across  the  final  3 sessions  of  a condition  for  each  trial  of 
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Table  8 cont. 
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Proportion  of  fixed  (F)  and  variable  (V)  choices  consistent  with  the  model’s  predictions  across  the  final  3 sessions  of  a condition  for  each  trial  of  a 
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Table  9 cont. 
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74 

energy-budget  conditions,  deviations  were  the  result  of  both  choices  for  the  fixed  option  when 
preference  of  the  variable  option  was  predicted,  and  variable  choices  when  preference  for  the  fixed 
option  was  predicted. 

Within-block  choice  patterns  were  evaluated  in  relation  to  the  canonical  cost  measures  by 
plotting  the  proportion  of  choices  consistent  with  predictions  as  a function  of  the  canonical  cost  of 
choosing  the  non-optimal  choice.  These  results  are  shown  in  Figure  10.  Overall,  choice  of 
Subjects  33 1 and  332  showed  that  choice  was  more  consistent  with  predictions  at  higher  than 
lower  canonical  costs.  This  trend  was  less  apparent  for  Subject  333.  For  this  subject,  the 
proportion  of  choices  consistent  with  predictions  showed  little  relation  to  canonical  cost. 

Discussion 

Choice  for  all  subjects  was  sensitive  to  energy-budget  conditions.  For  the  two  subjects 
showing  the  greatest  sensitivity  to  energy-budget  conditions  (Subjects  331  and  332),  choice  was 
risk  averse  during  positive  energy-budget  conditions  and  risk  prone  during  negative  energy-budget 
conditions.  Furthermore,  choice  was  more  risk  prone  during  the  Negative-Energy  Budget  (R=13) 
condition,  in  which  the  point  requirement  was  more  stringent,  than  during  the  Negative-Energy- 
Budget  (R=12)  condition.  For  Subject  333,  choice  was  risk  averse  during  two  of  the  three 
exposures  to  the  Positive  Energy-Budget  (R=10)  condition.  Choice  remained  risk  averse  during  the 
Negative-Energy  Budget  (R=  13)  conditions,  but  the  number  of  choices  for  the  fixed  option  was 
lower  than  during  the  Positive  Energy-Budget  (R=10)  conditions.  Thus,  although  choice  for 
Subject  333  did  not  show  complete  shifts  from  risk  aversion  to  risk  proneness  across  changes  in 
energy  budget,  choice  shifted  in  the  same  direction  as  in  the  other  two  subjects.  Choices  for 
Subject  333  were  therefore  qualitatively  consistent,  and  choices  for  Subjects  33 1 and  332  were 
quantitatively  consistent,  with  the  predictions  of  the  energy-budget  model. 
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Fig.  10.  Proportion  of  within-block  responses  across  the  final  three  sessions  of 
each  energy-budget  condition  that  were  consistent  with  the  predictions  of  an 
optimization  model  plotted  as  a function  of  the  canonical  cost  of  selecting  the  non- 
optimal  alternative.  Values  for  each  subject  are  shown. 
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The  within-block  choice  patterns  also  showed  sensitivity  to  the  energy-budget 
contingencies.  Choices  were  generally  consistent  with  the  predictions  of  a dynamic  optimization 
model,  especially  for  Subjects  33 1 and  332,  indicating  that  choices  were  optimal  not  only  at  the 
global  level  but  also  at  the  more  local  level  of  individual  choices.  Because  the  model’s  predictions 
were  determined  by  the  number  of  accumulated  points,  the  number  of  points  produced  by  fixed  and 
variable  choices,  the  position  within  the  block,  and  the  point  requirement,  these  results  show  that, 
for  at  least  two  subjects,  choice  was  simultaneously  sensitive  to  all  of  these  variables.  These 
results  also  showed  that  across-block  choice  patterns  in  positive  and  negative  energy-budget 
conditions  were  not  the  result  of  general  risk-averse  or  risk-prone  preferences,  but  were  the  result 
of  specific  within-block  choices. 

For  Subjects  331  and  332,  although  choices  occasionally  deviated  from  the  predictions  of 
the  optimization  model,  the  deviations  were  systematically  related  to  the  canonical-cost  measures. 
When  deviating  from  the  optimal  pattern  was  not  very  costly  (i.e.,  early  in  the  block),  the 
proportion  of  choices  consistent  with  predictions  was  lower  than  when  deviating  from  the  optimal 
pattern  was  more  costly  (i.e.,  later  in  the  block).  These  results  thus  suggest  that  within-block 
choice  patterns  were  sensitive  to  the  differential  (point)  consequences  of  selecting  the  fixed  and 
variable  option. 

Mean  point  earnings  per  session  also  showed  that  choice  patterns  frequently  maximized 
reinforcement  rates.  During  both  negative  energy-budget  conditions,  exclusive  preference  for  the 
variable  option  would  meet  the  requirement  with  p=.  1 9,  and  thus  produce  an  average  of  1 1 .3 
points.  As  a result  of  switching  between  fixed  and  variable  choices  during  negative  energy-budget 
conditions,  however,  point  earnings  were  often  higher  than  this  value,  especially  for  Subjects  33 1 
and  332.  For  example,  assuming  that  choice  would  switch  from  the  variable  to  the  fixed  option 
following  the  receipt  of  two,  3-point  variable  outcomes  during  Negative  Energy-Budget  (R-12) 
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conditions,  the  predicted  probability  of  exceeding  the  point  requirement  increases  to  p = .38. 
Similarly,  assuming  that  choice  would  switch  from  the  variable  to  the  fixed  option  following  the 
receipt  of  three,  3-point  variable  outcomes  during  Negative  Energy-Budget  (R=13)  conditions,  the 
predicted  probability  of  exceeding  the  point  requirement  increases  to  p=.22.  With  this  pattern  of 
switching,  during  the  Negative-Energy  Budget  (R=  1 2)  and  Negative-Energy  Budget  (R=13) 
conditions,  the  predicted  mean  point  earnings  increases  to  22.5  points  and  13.4,  respectively.  For 
two  subjects  (Subjects  33 1 and  332),  obtained  point  earnings  were  in  most  cases  closer  to  these 
values.  That  switching  between  alternatives  could  yield  higher  point  earnings  than  exclusive 
preferences  is  consistent  with  suggestions  made  by  behavioral  ecologists  that  switching  between 
fixed  and  variable  options  can  yield  higher  measures  of  fitness  (e.g.,  Houston  & McNamara, 

1982). 

The  present  results  are  analogous  to  those  shown  in  energy-budget  research  with 
nonhumans  (e.g.  Barnard  & Brown,  1985;  Caraco  et  al.,  1980).  Of  particular  relevance  to  the 
present  study  are  the  results  of  an  experiment  by  Caraco  et  al.  (1990).  In  their  study,  choice  was 
risk  averse  when  energy  requirements  were  low  (positive  energy-budget  conditions),  but  choice  was 
risk  prone  when  energy  requirements  were  high  (negative  energy-budget  conditions).  Similarly,  in 
the  present  study  choice  was  risk  averse  when  the  point  requirement  was  low  (positive  energy- 
budget  conditions),  and  more  risk  prone  when  the  point  requirement  was  high  (negative  energy- 
budget  conditions).  Additional  research  is  required  to  determine  whether  the  same  shifts  in  risk 
sensitivity  can  be  produced  in  humans  by  the  more  common  methods  of  changing  energy  budget: 
Changing  the  rate  of  reinforcement  (i.e.,  changing  the  point  earnings  on  the  fixed  and  variable 
option),  and  changing  energy  reserves  (i.e.,  changing  points  available  at  the  start  of  each  block). 

Several  points  concerning  the  use  of  dynamic  optimization  models  deserve  comment. 

First,  dynamic  optimization  models  have  been  criticized  on  several  grounds,  including  their 
complexity  (see  commentary  on  Houston  & McNamara,  1988).  For  example,  Smith  (1988)  has 
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noted  that  although  dynamic  models  can  offer  greater  realism  than  static  optimization  models, 
because  their  predictions  are  based  upon  multiple  independent  variables,  they  are  much  more 
difficult  to  test.  Testing  dynamic  optimization  models.  Smith  argued,  may  be  more  feasible  in 
operant  procedures.  The  analyses  presented  here  provide  some  additional  support  for  this 
statement.  That  is,  the  within-block  choice  analyses  showed  that  when  behavioral  contingencies 
are  well  controlled,  choices  could  be  readily  evaluated  in  relation  to  the  predictions  of  a dynamic 
model.  Second,  these  analyses  suggest  that  dynamic  optimization  models  may  be  useful  for 
operant  analyses  of  behavior.  In  the  present  experiment,  complex,  within-block  choice  patterns 
were  often  well  described  by  the  predictions  of  a dynamic  optimization  model  whose  main 
assumption  was  that  choices  would  maximize  reinforcement  rates.  Thus,  dynamic  optimization 
models  may  be  useful  for  describing  and  predicting  dynamic  behavior  patterns  resulting  from  the 
interaction  of  prior  consequences  and  current  reinforcement  contingencies.  Furthermore,  that 
deviations  from  optimal  choices  were  often  related  to  the  costs  of  selecting  the  non-optimal 
alternative  (canonical-cost  measures)  suggests  that  dynamic  models  may  be  used  to  predict  the 
conditions  under  which  non-optimal  behavior  may  occur.  That  is,  deviations  from  optimality  may 
be  more  likely  when  canonical  costs  are  low  than  when  they  are  high. 


GENERAL  DISCUSSION 


The  present  experiments  examined  the  generality  of  results  of  previous  risky-choice 
research  with  humans  by  investigating  choice  under  conditions  more  similar  to  those  used  with 
nonhumans.  Specifically,  subjects  were  given  relatively  extensive  exposure  to  choice  contingencies 
when  choices  produced  real  monetary  outcomes.  Experiments  1 and  2 examined  choice  between 
fixed  and  variable  reinforcer  amounts  and  delays,  respectively,  as  the  form  of  the  variable 
distribution  was  manipulated  across  conditions.  In  both  experiments,  choice  was  either  risk  averse 
or  risk  neutral.  These  results  are  generally  consistent  with  previous  research  with  humans, 
demonstrating  that  patterns  of  risk  sensitivity  obtained  in  studies  using  hypothetical  outcomes  and 
verbally  described  contingencies  generalize  to  situations  in  which  choice  outcomes  are  real  and 
choice  contingencies  are  actually  experienced. 

Experiment  3 examined  choice  between  fixed  and  variable  reinforcer  amounts  across 
experimental  manipulations  designed  to  model  energy-budget  manipulations  conducted  with 
nonhumans.  Unlike  the  results  of  the  first  two  experiments,  risk  sensitivity  varied  systematically 
across  conditions.  Choice  was  risk  averse  during  positive  energy-budget  conditions  but  frequently 
risk  prone  during  negative  energy-budget  conditions.  These  results  suggest  that  the  consistent  risk 
aversion  shown  in  previous  research  with  humans  may  be  due  in  part  to  the  procedures  typically 
used  to  assess  choice. 

A number  of  researchers  have  commented  on  the  advantages  of  studying  human  behavior 
under  conditions  resembling  those  used  with  nonhumans  (e.g.  Baron,  Perone,  & Galizio,  1991; 
Hackenberg,  1998;  Kollins,  Newland,  & Critchfield,  1997).  For  example,  Baron  et  al.  have 


79 


80 


argued  that  giving  human  subjects  prolonged  exposure  to  contingencies  (similar  to  studies  with 
nonhumans)  may  increase  control  of  behavior  by  current  reinforcement  contingencies  and  reduce 
control  by  extraneous  variables.  The  results  of  Experiments  1 and  2 provide  some  support  this 
assumption.  In  many  cases,  patterns  of  risk  sensitivity  obtained  during  initial  conditions  of  the 
experiment  were  more  variable  and  sometimes  very  different  than  patterns  obtained  during 
subsequent  conditions,  suggesting  that  experience  with  choice  contingencies  influenced  responding. 
If  choice  had  been  evaluated  across  a few  sessions  or  choice  trials,  the  conclusions  would  have 
been  very  different. 

Despite  the  extensive  exposure  to  choice  contingencies,  the  risk  sensitivity  obtained  in  the 
present  research  was  not  always  consistent  with  the  risk  sensitivity  shown  in  nonhumans.  That  is, 
although  the  risk  aversion  obtained  in  Experiment  1 was  comparable  with  the  results  of  some 
studies  with  nonhumans,  the  risk  aversion  and  risk  neutrality  shown  in  Experiment  2 contrasted 
with  the  strong  risk  proneness  shown  in  previous  studies  with  nonhumans.  This  finding  suggests 
that  other  procedural  differences  between  the  present  experiments  and  experiments  conducted  with 
nonhumans  may  be  important. 

One  procedural  variable  that  may  have  contributed  to  the  differences  in  responding 
concerns  the  nature  of  the  reinforcer.  Although  the  present  research  used  real  money  as  an 
outcome,  it  is  likely  that  delays  to  money  (a  nonconsumable  reinforcer)  do  not  have  the  same  effect 
on  choice  as  delays  to  food  (a  consumable  reinforcer).  Previous  research  has  shown  that  when 
presented  with  choices  between  a small  amount  of  food  delivered  after  a delay,  nonhumans  strongly 
prefer  the  immediate  option  (e.g.,  Rachlin  & Green,  1972).  Analogous  studies  with  humans  using 
monetary  reinforcers  have  shown  the  opposite  effect  (e.g.,  Logue,  Pena-Correal,  Rodriguez,  & 
Kabela,  1986).  Several  experiments  with  humans  using  consumable  reinforcers,  however,  such  as 
food  deliveries  (Forzano  & Logue,  1994;  Logue  & King,  1991)  or  access  to  a pre-recorded  TV 
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program  (Navarick,  1996),  have  shown  greater  sensitivity  to  reinforcer  immediacy.  For  example,  a 
recent  study  in  our  laboratory  using  access  to  a pre-recorded  TV  program  as  a reinforcer  showed 
that  adult  humans  preferred  more  immediate  access  to  a smaller  reinforcer  (a  20-s  video  segment) 
to  delayed  access  to  a larger  reinforcer  (a  60-s  video  segment)  (Hackenberg  & Pietras,  2000). 
Together,  these  studies  suggest  that  greater  delay  discounting  occurs  when  choice  outcomes  are 
immediately  consumable  reinforcers  than  when  they  are  nonconsumable  reinforcers.  Because  delay 
discounting  is  thought  to  underlie  the  preference  for  variable  over  fixed  delays  to  reinforcement  in 
nonhumans,  it  is  possible  that  choice  of  humans  would  also  show  greater  risk  proneness  when 
choice  outcomes  are  consumable  reinforcers. 

Although  investigating  choice  in  humans  with  consumable  reinforcers  is  an  important 
direction  for  future  risky-choice  research,  there  are  a number  of  advantages  to  studying  human  risk 
taking  using  monetary  reinforcers.  Many  examples  of  human  risk  taking  outside  the  laboratory 
involve  monetary  reinforcers  (e.g.,  gambling  or  playing  the  stock  market).  Using  monetary 
reinforcers  may  therefore  facilitate  the  development  of  laboratory  models  that  generalize  more 
readily  to  circumstances  outside  the  laboratory.  In  addition,  results  of  experiments  using  monetary 
reinforcers  can  be  integrated  with  a substantial  amount  of  risky-choice  research  conducted  by 
economists,  cognitive  psychologists,  and  researchers  interested  in  decision  making,  the  majority  of 
which  has  been  concerned  with  monetary  outcomes.  There  are  also  many  practical  advantages  to 
using  money  as  a reinforcer.  No  specific  deprivation  procedures  are  required  to  establish  money  as 
an  effective  reinforcer,  and  monetary  reinforcers  are  easy  to  manipulate  and  deliver.  Thus,  it  is 
important  to  develop  procedures  that  can  produce  risk  proneness  with  monetary  outcomes. 

The  energy-budget  procedure  developed  in  Experiment  3 was  shown  to  be  such  a 
procedure.  Unlike  manipulations  conducted  in  Experiments  1 and  2,  the  energy-budget 
manipulations  produced  both  risk  aversion  and  risk  proneness  with  monetary  reinforcers. 
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Furthermore,  the  shifts  in  risk  sensitivity  occurred  within  individual  subjects  and  were  replicated 
across  conditions.  These  results  have  several  implications  for  human  risky  choice  and  the  cross- 
species generality  of  risky  choice.  Previous  research  has  shown  that  choice  in  humans  is  more 
consistently  risk  averse  than  choice  in  nonhumans.  The  reliable  patterns  of  risk  sensitivity  obtained 
here  implies,  however,  that  past  human-nonhuman  differences  may  have  less  to  do  with  species 
differences  than  with  differences  in  the  procedures  used  to  assess  choice  in  humans  and  other 
animals.  At  least  in  the  present  case,  humans’  choices  were  shown  to  be  similar  to  nonhumans’ 
choices  when  behavior  was  studied  under  comparable  experimental  conditions. 

That  energy-budgets  and  “eamings-budgets”  have  similar  effects  on  choice  suggests  that 
the  patterns  of  risk  sensitivity  generated  by  the  relationship  between  the  gain  from  fixed  and 
variable  choices,  current  state,  temporal  constraints,  and  requirements,  extend  beyond  those 
situations  involving  biologically  important  consequences.  Although  based  upon  assumptions  about 
the  fitness  consequences  of  risk-averse  and  risk-prone  choices  for  nonhuman  foragers,  the 
predictions  of  the  energy-budget  rule  thus  generalize  to  a broad  range  of  contexts.  Not  only  can 
the  predictions  be  applied  to  risky  choice  in  humans,  but  they  can  be  applied  to  situations  in  which 
outcomes  are  monetary  reinforcers. 

Because  one  goal  of  the  present  research  was  to  investigate  choice  in  humans  under 
conditions  similar  to  those  used  with  nonhumans,  research  and  models  developed  by  behavioral 
ecologists  have  been  emphasized.  The  results  are  also  relevant,  however,  to  models  developed  by 
cognitive  psychologists.  Of  particular  note  is  the  finding,  common  in  judgment  and  decision- 
making research,  that  risky  choice  in  humans  is  influenced  by  the  probability  that  a choice  outcome 
will  fall  above  or  below  a particular  monetary  value  (e.g.,  Payne,  Laughhunn,  Crum,  1980,  1981; 
Tversky  & Kahneman,  1981).  This  value  has  been  called  a target,  reference  point,  or  aspiration 
level,  and  is  used  to  explain  the  well-documented  finding  of  risk  aversion  when  choice  outcomes 
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are  gains  and  risk  proneness  when  outcomes  are  losses  (Kahneman  & Tversky,  1979).  Caraco  and 
Lima  (1987)  have  observed  that  the  shift  from  risk  aversion  to  risk  proneness  shown  in  nonhumans 
across  positive  and  negative  energy  budgets  is  similar  to  the  shift  from  risk  aversion  to  risk 
proneness  shown  in  humans  across  monetary  gains  and  losses. 

As  noted  by  Luce  (1996),  however,  researchers  lack  suitable  methods  for  estimating 
targets  in  individual  subjects.  The  present  procedure  eliminates  this  problem  by  specifically 
establishing  targets  within  the  experimental  setting.  Furthermore,  because  the  present  procedures 
combined  the  monetary  outcomes  typical  of  cognitive  experiments  on  decision-making  with  energy- 
budget  manipulations  used  optimal  foraging  research,  they  help  establish  a more  direct  link 
between  models  developed  by  psychologists  and  those  developed  by  behavioral  ecologists. 

To  date,  there  have  been  few  attempts  to  integrate  risky-choice  research  conducted  by 
operant  psychologists,  cognitive  psychologists,  and  behavioral  ecologists  into  a single  conceptual 
framework.  A major  obstacle  to  the  development  of  an  interdisciplinary  approach  to  risky  choice 
is  the  vastly  different  methods  employed  by  researchers  in  different  traditions.  For  example,  it  is 
unclear  how  models  which  assume  that  repeated  choices  under  steady-state  conditions  are  governed 
by  the  amount  or  the  immediacy  of  a primary  reinforcer  can  be  applied  to  the  results  of  studies 
with  humans  using  one-shot  choices  and  hypothetical  monetary  amounts.  Neither  is  it  clear  how 
models  which  assume  that  risky  choice  is  determined  by  verbally  presented  gambles  can  be  applied 
to  foraging-related  choices  in  nonhumans.  Developing  a common  set  of  procedures  that  can  be 
adopted  by  researchers  from  different  scientific  disciplines  may  help  promote  the  development  of 
more  general  models  and  interpretations  of  risky  choice  which  apply  across  a variety  of  species  and 


choice  contexts. 


APPENDIX  A 


WITHIN-BLOCK  CHOICES  FOR  EACH  SUBJECT  IN  EXPERIMENT  3 

Shown  below  are  the  number  of  fixed  (F)  and  variable  (V)  choices  for  each  trial  of  a block 
and  number  of  accumulated  points  for  each  subject  across  the  final  three  sessions  of  a condition 
(18  blocks).  The  total  the  total  number  of  choices  per  trial  thus  equaled  1 8.  The  optimal  choice 
for  each  trial  and  number  of  accumulated  points  is  underlined.  For  Subject  333,  only  results  from 
the  final  three  conditions  are  presented. 
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Subject  331 
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Subject  33 1 cont. 
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Subject  332  cont. 
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Subject  333  cont. 


90 


13 

■ 

H 


o m 


~ r?  CM 


i- 

3 

t/5 

G 

CL 

X 

a> 

TJ 

§ 

8 

c/i 

m 


^r 

13 

*c 

H 


m 

13 


V)  ^IO 


n ^ o ^ 


m oi 


c*->  ~ — ~ 


3 

f 

Dd 


CM 

13 

*C 

H 


ml  Tf  —| 


(N  VO  (N 


*n 

H 


ool 


t/5 

M 

C 

o 

a. 


APPENDIX  B 


GENERATING  PREDICTIONS  WITH  A DYNAMIC  OPTIMIZATION  MODEL 

To  illustrate  how  the  predictions  of  the  dynamic  optimization  model  were  generated,  it  will 
be  useful  to  work  through  an  example  similar  to  one  provided  by  Houston  and  McNamara  (1988). 
Consider  a choice  during  the  fifth  (final)  trial  of  a block  ( T-l ) if  8 points  have  been  accumulated 
(x=8)  and  the  requirement  was  10  points.  As  described  in  the  text,  the  terminal  fitness  function, 
R(x),  was  defined  as: 


Selections  of  the  fixed  option  produce  2 points  and  selections  of  the  variable  option  produce  either 
1 or  3 points  (p=.5).  If  the  variable  option  is  selected,  the  final  value  of  x will  be  either  9 or  1 1 
with  equal  probability.  If  the  fixed  option  is  selected,  the  final  value  of  x will  be  10  with  certainty. 
Thus,  the  expected  earnings,  E,  at  state  x,  given  a fixed  (F)  or  variable  (V)  choice,  at  time  T-l  may 
be  calculated  as: 


E{ 8,  F,  T-/)  = (l  */?(10))  = (l  * 10)=  10 


£(8,  V,  T-l)  = (.5  * R{ 9))  + (.5  * /?(1 1))  = (.5  * 0)  + (.5  * 1 1)  = 5.5 
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Thus,  the  optimal  choice  is  the  fixed  option.  However,  if  7 points  had  been  accumulated  at  T-l 
then 


E( 7,  F,  T-l)  = (1  * R( 9))  = (1  * 0)  = 0 

E(7,  V,  T-l)  = (.5  * R( 8))  + (.5  * fl(10))  = (.5  * 0)  + (.5  * 10)  = 5 

in  which  case  the  optimal  choice  is  the  variable  option.  To  predict  choice  during  the  fourth  trial  of 
a block  (T-2),  the  values  of  the  terminal  reward  function,  R(x),  are  replaced  with  the  expected 
earnings  at  each  state  at  T-l,  designated  yApc),  given  that  the  optimal  choice  occurred.  For 
example,  because  choice  of  the  fixed  option  was  optimal  at  x=8  and  T-l,  ^(8)  at  T-2  will  equal  10. 
Thus,  if  6 points  had  been  accumulated  (x=6)  at  T-2: 


£(6,  F,  r-2)  = ( 1 * ^8,7W))  = (1  * 10)=  10 

E(6,  V,  T-2)  = (.5  * i/^7, T-l))  + (.5  * y/(9,T-l))  = (.5  * 5)  + (.5  * 1 1)  = 8 


Therefore,  the  optimal  choice  is  the  fixed  option.  At  each  time  and  state,  the  expected  earnings  are 
thus  the  average  number  of  points  that  will  be  accumulated  at  the  end  of  the  block  given  that  the 
optimal  choice  is  selected  on  each  remaining  trial. 
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